[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:56:51 dist-test-slave-6ntk krb5kdc[836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:56:51 dist-test-slave-6ntk krb5kdc[836](info): set up 2 sockets
Aug 08 02:56:51 dist-test-slave-6ntk krb5kdc[836](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 02:56:53.041495 822 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:56:53 dist-test-slave-6ntk krb5kdc[836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:56:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:56:53.242595 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43465
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:43465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:56:53.623374 859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:56:53.623811 859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:56:53.624042 859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:56:53.631634 859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:56:53.631771 859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:56:53.631891 859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:56:53.631994 859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:56:53.632091 859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:56:53.642958 859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:43465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:56:53.644966 859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:56:53.646932 859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:56:53.665638 868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:53.665638 867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:53.665515 870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:56:53.668037 859 server_base.cc:1034] running on GCE node
I20260808 02:56:53.669663 859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:56:53.672080 859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:56:53.673422 859 hybrid_clock.cc:648] HybridClock initialized: now 1786157813673302 us; error 102 us; skew 500 ppm
I20260808 02:56:53.674064 859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:56:53 dist-test-slave-6ntk krb5kdc[836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:56:53.677699 859 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:56:53.680176 859 webserver.cc:460] Webserver started at http://127.0.205.190:43259/ using document root <none> and password file <none>
I20260808 02:56:53.681113 859 fs_manager.cc:359] Metadata directory not provided
I20260808 02:56:53.681277 859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:56:53.681615 859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:56:53.685353 859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "60db296d32e24edc8d13cc7c9bca4e86"
format_stamp: "Formatted at 2026-08-08 02:56:53 on dist-test-slave-6ntk"
server_key: "0dedfebcf57a4e03760440ef22f63c38"
server_key_iv: "1cc13a9098bac38e643b08e18db48ea1"
server_key_version: "encryptionkey@0"
I20260808 02:56:53.686251 859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "60db296d32e24edc8d13cc7c9bca4e86"
format_stamp: "Formatted at 2026-08-08 02:56:53 on dist-test-slave-6ntk"
server_key: "0dedfebcf57a4e03760440ef22f63c38"
server_key_iv: "1cc13a9098bac38e643b08e18db48ea1"
server_key_version: "encryptionkey@0"
I20260808 02:56:53.692741 859 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 02:56:53.696815 879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:53.698395 859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:56:53.698700 859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "60db296d32e24edc8d13cc7c9bca4e86"
format_stamp: "Formatted at 2026-08-08 02:56:53 on dist-test-slave-6ntk"
server_key: "0dedfebcf57a4e03760440ef22f63c38"
server_key_iv: "1cc13a9098bac38e643b08e18db48ea1"
server_key_version: "encryptionkey@0"
I20260808 02:56:53.698926 859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:56:53.735395 859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:56:53.741331 859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:56:53.741729 859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:56:53.765547 859 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:43465
I20260808 02:56:53.765566 957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:43465 every 8 connection(s)
I20260808 02:56:53.767983 859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:56:53.774338 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 859
I20260808 02:56:53.774827 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:56:53.775629 958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:56:53.778234 822 external_mini_cluster.cc:1428] Setting key 27c7d496df5064295c2e6ac508dc1612
I20260808 02:56:53.797274 958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86: Bootstrap starting.
I20260808 02:56:53.804137 958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86: Neither blocks nor log segments found. Creating new log.
I20260808 02:56:53.806221 958 log.cc:824] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86: Log is configured to *not* fsync() on all Append() calls
I20260808 02:56:53.810900 958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86: No bootstrap required, opened a new log
Aug 08 02:56:53 dist-test-slave-6ntk krb5kdc[836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:56:53.823361 958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60db296d32e24edc8d13cc7c9bca4e86" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43465 } }
I20260808 02:56:53.823936 958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:56:53.824165 958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db296d32e24edc8d13cc7c9bca4e86, State: Initialized, Role: FOLLOWER
I20260808 02:56:53.825366 958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60db296d32e24edc8d13cc7c9bca4e86" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43465 } }
I20260808 02:56:53.825748 958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:56:53.826093 958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:56:53.826335 958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:56:53.829561 958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60db296d32e24edc8d13cc7c9bca4e86" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43465 } }
I20260808 02:56:53.830590 958 leader_election.cc:302] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60db296d32e24edc8d13cc7c9bca4e86; no voters:
I20260808 02:56:53.831622 958 leader_election.cc:288] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:56:53.831897 971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:56:53.833923 971 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [term 1 LEADER]: Becoming Leader. State: Replica: 60db296d32e24edc8d13cc7c9bca4e86, State: Running, Role: LEADER
I20260808 02:56:53.834478 964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:53.788041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41624 (local address 127.0.205.190:43465)
0808 02:56:53.789286 (+ 1245us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:53.789299 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:53.789349 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:56:53.804100 (+ 14751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:53.804127 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:53.804796 (+ 669us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:53.805193 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:53.810011 (+ 4818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:53.811438 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:53.814326 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:53.815050 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:53.815258 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:53.822996 (+ 7738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:53.823070 (+ 74us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:53.823106 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:53.823156 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:53.826876 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:53.828013 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:53.828020 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:53.828026 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:53.828253 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:53.828860 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:53.828865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:53.828868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:53.829692 (+ 824us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:53.830444 (+ 752us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:53.831685 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0808 02:56:53.832168 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":938,"thread_start_us":587,"threads_started":1}
I20260808 02:56:53.835058 971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60db296d32e24edc8d13cc7c9bca4e86" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43465 } }
I20260808 02:56:53.837990 958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:56:53.841332 974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60db296d32e24edc8d13cc7c9bca4e86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60db296d32e24edc8d13cc7c9bca4e86" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43465 } } }
I20260808 02:56:53.841374 972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60db296d32e24edc8d13cc7c9bca4e86. Latest consensus state: current_term: 1 leader_uuid: "60db296d32e24edc8d13cc7c9bca4e86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60db296d32e24edc8d13cc7c9bca4e86" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43465 } } }
I20260808 02:56:53.841900 972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [sys.catalog]: This master's current role is: LEADER
I20260808 02:56:53.841965 974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86 [sys.catalog]: This master's current role is: LEADER
I20260808 02:56:53.843173 981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:56:53.860735 981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:56:53.878048 981 catalog_manager.cc:1347] Generated new cluster ID: c731e2ca8a8e4fe7a68665a3740555c5
I20260808 02:56:53.878162 981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:56:53.900520 981 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:56:53.902134 981 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:56:53.910632 981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60db296d32e24edc8d13cc7c9bca4e86: Generated new TSK 0
I20260808 02:56:53.911974 981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:56:53.981638 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43465
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:56:54.354737 1008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:56:54.355021 1008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:56:54.355120 1008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:56:54.362358 1008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:56:54.362470 1008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:56:54.362588 1008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:56:54.372575 1008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43465
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:56:54.373950 1008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:56:54.375227 1008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:56:54.389103 1023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:54.389253 1020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:54.389125 1021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:56:54.390571 1008 server_base.cc:1034] running on GCE node
I20260808 02:56:54.391180 1008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:56:54.392138 1008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:56:54.393390 1008 hybrid_clock.cc:648] HybridClock initialized: now 1786157814393330 us; error 55 us; skew 500 ppm
I20260808 02:56:54.393734 1008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:56:54 dist-test-slave-6ntk krb5kdc[836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:56:54.397100 1008 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:56:54.398792 1008 webserver.cc:460] Webserver started at http://127.0.205.129:38479/ using document root <none> and password file <none>
I20260808 02:56:54.399521 1008 fs_manager.cc:359] Metadata directory not provided
I20260808 02:56:54.399679 1008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:56:54.400018 1008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:56:54.403156 1008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "181f8a1ae6854158b233c012ff1586c3"
format_stamp: "Formatted at 2026-08-08 02:56:54 on dist-test-slave-6ntk"
server_key: "bcbcc3afbcbbfe91ad6d3216f6bf8704"
server_key_iv: "d9a2fa9072e7ec69986e1fe5dc90df40"
server_key_version: "encryptionkey@0"
I20260808 02:56:54.404003 1008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "181f8a1ae6854158b233c012ff1586c3"
format_stamp: "Formatted at 2026-08-08 02:56:54 on dist-test-slave-6ntk"
server_key: "bcbcc3afbcbbfe91ad6d3216f6bf8704"
server_key_iv: "d9a2fa9072e7ec69986e1fe5dc90df40"
server_key_version: "encryptionkey@0"
I20260808 02:56:54.409559 1008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 02:56:54.413466 1033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:54.415115 1008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 02:56:54.415302 1008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "181f8a1ae6854158b233c012ff1586c3"
format_stamp: "Formatted at 2026-08-08 02:56:54 on dist-test-slave-6ntk"
server_key: "bcbcc3afbcbbfe91ad6d3216f6bf8704"
server_key_iv: "d9a2fa9072e7ec69986e1fe5dc90df40"
server_key_version: "encryptionkey@0"
I20260808 02:56:54.415526 1008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:56:54.437230 1008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:56:54.441669 1008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:56:54.442056 1008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:56:54.443393 1008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:56:54.445169 1008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:56:54.445281 1008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:54.445498 1008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:56:54.445585 1008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:54.485021 1008 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:34293
I20260808 02:56:54.485083 1210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:34293 every 8 connection(s)
I20260808 02:56:54.486541 1008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:56:54.496258 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1008
I20260808 02:56:54.496547 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:56:54.497305 822 external_mini_cluster.cc:1428] Setting key 9696e9859691d4bb8747183cdc95ad2e
Aug 08 02:56:54 dist-test-slave-6ntk krb5kdc[836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:56:54.508967 1214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:54.489903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:54319 (local address 127.0.205.190:43465)
0808 02:56:54.490374 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:54.490379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:54.491456 (+ 1077us) server_negotiation.cc:407] Connection header received
0808 02:56:54.493492 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:54.493498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:54.493595 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:54.493723 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:54.495392 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:54.496127 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:54.498035 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:54.498604 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:54.498748 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:54.502807 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:54.502814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:54.502818 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:54.502862 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:54.505585 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:54.506448 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:54.506452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:54.506455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:54.506582 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:54.507056 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:54.507060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:54.507062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:54.507438 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:54.507602 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:54.508337 (+ 735us) server_negotiation.cc:299] Negotiation successful
0808 02:56:54.508592 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":352,"thread_start_us":229,"threads_started":1}
I20260808 02:56:54.510145 1215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:54.490518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43465 (local address 127.0.205.129:54319)
0808 02:56:54.491273 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0808 02:56:54.491312 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:56:54.493156 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:56:54.494032 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:56:54.494041 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:56:54.494588 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:56:54.495186 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:54.495202 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:54.496296 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:54.496302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:54.497831 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:54.497844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:54.498853 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:54.498860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:54.499163 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:54.499910 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:56:54.499945 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:56:54.502554 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:56:54.505736 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:54.505746 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:54.505764 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:54.506298 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:54.506699 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:54.506704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:54.506708 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:54.506921 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:54.507576 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:56:54.507585 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:56:54.507957 (+ 372us) client_negotiation.cc:769] Sending connection context
0808 02:56:54.508237 (+ 280us) client_negotiation.cc:240] Negotiation successful
0808 02:56:54.508792 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":309,"threads_started":1}
I20260808 02:56:54.511976 1211 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43465
I20260808 02:56:54.512534 1211 heartbeater.cc:464] Registering TS with master...
I20260808 02:56:54.513394 1211 heartbeater.cc:511] Master 127.0.205.190:43465 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
I20260808 02:56:54.517320 902 ts_manager.cc:194] Registered new tserver with Master: 181f8a1ae6854158b233c012ff1586c3 (127.0.205.129:34293)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
I20260808 02:56:54.519965 902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:54319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:56:54.545682 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43465
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:56:54.912338 1219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:56:54.912657 1219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:56:54.912832 1219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:56:54.920655 1219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:56:54.920773 1219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:56:54.920981 1219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:56:54.931181 1219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43465
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:56:54.933079 1219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:56:54.934617 1219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:56:54.948717 1230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:54.948805 1234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:54.948699 1227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:56:54.950497 1219 server_base.cc:1034] running on GCE node
I20260808 02:56:54.951179 1219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:56:54.952225 1219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:56:54.953562 1219 hybrid_clock.cc:648] HybridClock initialized: now 1786157814953520 us; error 56 us; skew 500 ppm
I20260808 02:56:54.954031 1219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:56:54 dist-test-slave-6ntk krb5kdc[836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:56:54.957289 1219 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:56:54.959026 1219 webserver.cc:460] Webserver started at http://127.0.205.130:45663/ using document root <none> and password file <none>
I20260808 02:56:54.959769 1219 fs_manager.cc:359] Metadata directory not provided
I20260808 02:56:54.959928 1219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:56:54.960270 1219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:56:54.963480 1219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "d16c018924464d6ebc1eabc75cf22277"
format_stamp: "Formatted at 2026-08-08 02:56:54 on dist-test-slave-6ntk"
server_key: "ceaab439e44d71ec866fe67a8aedf918"
server_key_iv: "326884d1c8570fb4ac1f3997f7ede59f"
server_key_version: "encryptionkey@0"
I20260808 02:56:54.964311 1219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "d16c018924464d6ebc1eabc75cf22277"
format_stamp: "Formatted at 2026-08-08 02:56:54 on dist-test-slave-6ntk"
server_key: "ceaab439e44d71ec866fe67a8aedf918"
server_key_iv: "326884d1c8570fb4ac1f3997f7ede59f"
server_key_version: "encryptionkey@0"
I20260808 02:56:54.969630 1219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:56:54.973284 1241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:54.974721 1219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:56:54.974905 1219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "d16c018924464d6ebc1eabc75cf22277"
format_stamp: "Formatted at 2026-08-08 02:56:54 on dist-test-slave-6ntk"
server_key: "ceaab439e44d71ec866fe67a8aedf918"
server_key_iv: "326884d1c8570fb4ac1f3997f7ede59f"
server_key_version: "encryptionkey@0"
I20260808 02:56:54.975116 1219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:56:55.014189 1219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:56:55.018649 1219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:56:55.018960 1219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:56:55.019995 1219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:56:55.021445 1219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:56:55.021560 1219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:55.021708 1219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:56:55.021783 1219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:55.058647 1219 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:33989
I20260808 02:56:55.058696 1408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:33989 every 8 connection(s)
I20260808 02:56:55.060042 1219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:56:55.067725 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1219
I20260808 02:56:55.067955 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:56:55.068517 822 external_mini_cluster.cc:1428] Setting key e4809e13ce675bc6ac45cc50a0c7d332
Aug 08 02:56:55 dist-test-slave-6ntk krb5kdc[836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:56:55.084645 1414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:55.063195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:42555 (local address 127.0.205.190:43465)
0808 02:56:55.063674 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:55.063679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:55.064932 (+ 1253us) server_negotiation.cc:407] Connection header received
0808 02:56:55.066847 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:55.066852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:55.066940 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:55.067041 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:55.069134 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.070022 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.072345 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.072803 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.072923 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:55.077364 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:55.077370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:55.077373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:55.077422 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:55.081029 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.081826 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.081835 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.081841 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.082060 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.082509 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.082516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.082522 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.082950 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:55.083100 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:55.083841 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 02:56:55.084122 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":254,"threads_started":1}
I20260808 02:56:55.085327 1415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:55.063907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43465 (local address 127.0.205.130:42555)
0808 02:56:55.064631 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 02:56:55.064672 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:56:55.066583 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:56:55.067243 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:56:55.067251 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:56:55.068096 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:56:55.068948 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:55.068975 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.070172 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.070176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:55.072180 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:55.072200 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.073047 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.073057 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:55.073439 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:55.074289 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:56:55.074324 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:56:55.077128 (+ 2804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:56:55.081177 (+ 4049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:55.081185 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:55.081200 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:55.081678 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:55.082186 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:55.082190 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:55.082192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:55.082385 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:55.083091 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:56:55.083098 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:56:55.083450 (+ 352us) client_negotiation.cc:769] Sending connection context
0808 02:56:55.083700 (+ 250us) client_negotiation.cc:240] Negotiation successful
0808 02:56:55.084142 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":290,"threads_started":1}
I20260808 02:56:55.087031 1409 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43465
I20260808 02:56:55.087374 1409 heartbeater.cc:464] Registering TS with master...
I20260808 02:56:55.088195 1409 heartbeater.cc:511] Master 127.0.205.190:43465 requested a full tablet report, sending...
I20260808 02:56:55.089829 902 ts_manager.cc:194] Registered new tserver with Master: d16c018924464d6ebc1eabc75cf22277 (127.0.205.130:33989)
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
I20260808 02:56:55.090775 902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:42555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:56:55.121284 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43465
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:56:55.472949 1422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:56:55.473371 1422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:56:55.473582 1422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:56:55.481441 1422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:56:55.481573 1422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:56:55.481765 1422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:56:55.495081 1422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43465
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:56:55.496430 1422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:56:55.497839 1422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:56:55.512380 1431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:55.512732 1435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:55.512784 1433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:56:55.513711 1422 server_base.cc:1034] running on GCE node
I20260808 02:56:55.514411 1422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:56:55.515362 1422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:56:55.516634 1422 hybrid_clock.cc:648] HybridClock initialized: now 1786157815516582 us; error 57 us; skew 500 ppm
I20260808 02:56:55.517005 1422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:56:55 dist-test-slave-6ntk krb5kdc[836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157815, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:56:55.520453 1422 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:56:55.522089 1422 webserver.cc:460] Webserver started at http://127.0.205.131:41319/ using document root <none> and password file <none>
I20260808 02:56:55.522763 1422 fs_manager.cc:359] Metadata directory not provided
I20260808 02:56:55.522869 1422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:56:55.523146 1422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:56:55.523794 1211 heartbeater.cc:503] Master 127.0.205.190:43465 was elected leader, sending a full tablet report...
I20260808 02:56:55.526324 1422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "afbbdc45eac1477bb6156d5bf36ec3f0"
format_stamp: "Formatted at 2026-08-08 02:56:55 on dist-test-slave-6ntk"
server_key: "d61295c898b4aabcab0648b8b3fe3230"
server_key_iv: "d95f9b2a68abb857a3d26cbdbe26ef4a"
server_key_version: "encryptionkey@0"
I20260808 02:56:55.527184 1422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "afbbdc45eac1477bb6156d5bf36ec3f0"
format_stamp: "Formatted at 2026-08-08 02:56:55 on dist-test-slave-6ntk"
server_key: "d61295c898b4aabcab0648b8b3fe3230"
server_key_iv: "d95f9b2a68abb857a3d26cbdbe26ef4a"
server_key_version: "encryptionkey@0"
I20260808 02:56:55.532763 1422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 02:56:55.536700 1450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:55.538196 1422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:56:55.538375 1422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "afbbdc45eac1477bb6156d5bf36ec3f0"
format_stamp: "Formatted at 2026-08-08 02:56:55 on dist-test-slave-6ntk"
server_key: "d61295c898b4aabcab0648b8b3fe3230"
server_key_iv: "d95f9b2a68abb857a3d26cbdbe26ef4a"
server_key_version: "encryptionkey@0"
I20260808 02:56:55.538591 1422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:56:55.568629 1422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:56:55.573660 1422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:56:55.574069 1422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:56:55.575147 1422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:56:55.576773 1422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:56:55.576885 1422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:55.577050 1422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:56:55.577138 1422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:55.618305 1422 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:32849
I20260808 02:56:55.618316 1618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:32849 every 8 connection(s)
I20260808 02:56:55.619736 1422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:56:55.622721 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1422
I20260808 02:56:55.622951 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:56:55.623526 822 external_mini_cluster.cc:1428] Setting key fc38bfe2b29e8096812c629299d4181a
Aug 08 02:56:55 dist-test-slave-6ntk krb5kdc[836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157815, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:56:55.643844 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:55.622841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:53167 (local address 127.0.205.190:43465)
0808 02:56:55.623346 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:55.623353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:55.624518 (+ 1165us) server_negotiation.cc:407] Connection header received
0808 02:56:55.626823 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:55.626827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:55.626909 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:55.627003 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:55.628773 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.629615 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.631638 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.632044 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.632142 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:55.636355 (+ 4213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:55.636361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:55.636364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:55.636406 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:55.639989 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.640868 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.640873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.640876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.641021 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.641606 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.641618 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.641620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.642032 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:55.642240 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:55.643070 (+ 830us) server_negotiation.cc:299] Negotiation successful
0808 02:56:55.643342 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":235,"threads_started":1}
I20260808 02:56:55.644927 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:55.623439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43465 (local address 127.0.205.131:53167)
0808 02:56:55.624327 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 02:56:55.624375 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:56:55.626451 (+ 2076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:56:55.627387 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:56:55.627399 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:56:55.628016 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:56:55.628608 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:55.628625 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.629814 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.629819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:55.631490 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:55.631506 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.632305 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.632312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:55.632568 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:55.633331 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:56:55.633358 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:56:55.636047 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:56:55.640179 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:55.640189 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:55.640207 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:55.640703 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:55.641162 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:55.641168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:55.641173 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:55.641453 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:55.642192 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:56:55.642201 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:56:55.642620 (+ 419us) client_negotiation.cc:769] Sending connection context
0808 02:56:55.642956 (+ 336us) client_negotiation.cc:240] Negotiation successful
0808 02:56:55.643565 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":416,"threads_started":1}
I20260808 02:56:55.646504 1619 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43465
I20260808 02:56:55.646872 1619 heartbeater.cc:464] Registering TS with master...
I20260808 02:56:55.647766 1619 heartbeater.cc:511] Master 127.0.205.190:43465 requested a full tablet report, sending...
I20260808 02:56:55.649175 902 ts_manager.cc:194] Registered new tserver with Master: afbbdc45eac1477bb6156d5bf36ec3f0 (127.0.205.131:32849)
I20260808 02:56:55.650066 902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:53167
I20260808 02:56:55.653456 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:56:55 dist-test-slave-6ntk krb5kdc[836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157815, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:56:55 dist-test-slave-6ntk krb5kdc[836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157815, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:56:55.705056 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:55.688709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54620 (local address 127.0.205.129:34293)
0808 02:56:55.689640 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:55.689648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:55.689678 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:56:55.689742 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:55.689748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:55.690058 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:55.690299 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:55.691771 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.692844 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.694205 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.694820 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.694920 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:55.699148 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:55.699161 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:55.699175 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:55.699210 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:55.702055 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.702800 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.702816 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.702818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.702946 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.703363 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.703369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.703372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.703759 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:55.704117 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:55.704424 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 02:56:55.704700 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":567,"threads_started":1}
W20260808 02:56:55.706588 1080 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54620
Aug 08 02:56:55 dist-test-slave-6ntk krb5kdc[836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:56:55 dist-test-slave-6ntk krb5kdc[836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:56:55.758570 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:55.743447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.0.205.129:34293)
0808 02:56:55.743682 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:55.743690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:55.744001 (+ 311us) server_negotiation.cc:407] Connection header received
0808 02:56:55.744237 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:55.744241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:55.744339 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:55.744465 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:55.745923 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.746708 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.748090 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.748509 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.748645 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:55.753049 (+ 4404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:55.753055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:55.753058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:55.753097 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:55.755814 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.756532 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.756538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.756543 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.756672 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.757052 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.757057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.757063 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.757326 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:55.757591 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:55.757964 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 02:56:55.758207 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:56:55.760597 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1008
I20260808 02:56:55.775259 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:55.761711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.0.205.129:34293)
0808 02:56:55.761966 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:55.761973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:55.762235 (+ 262us) server_negotiation.cc:407] Connection header received
0808 02:56:55.762358 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:55.762362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:55.762430 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:55.762507 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:55.763666 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.764369 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.766295 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.766760 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.766855 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:55.769673 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:55.769678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:55.769681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:55.769712 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:55.772506 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.773250 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.773254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.773256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.773486 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.773967 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.773971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.773973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.774219 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:55.774413 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:55.774760 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 02:56:55.774925 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:56:55.776150 1206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:56:55.865445 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1008
I20260808 02:56:55.888278 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1219
Aug 08 02:56:55 dist-test-slave-6ntk krb5kdc[836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:56:55.905170 1640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:55.889194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33918 (local address 127.0.205.130:33989)
0808 02:56:55.889766 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:55.889774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:55.889799 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:56:55.889912 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:55.889928 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:55.890180 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:55.890354 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:55.891625 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.892701 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.894180 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:55.894772 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:55.894929 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:55.899191 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:55.899203 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:55.899219 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:55.899264 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:55.902047 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.902788 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.902800 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.902803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.902944 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:55.903393 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:55.903400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:55.903407 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:55.903797 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:55.904156 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:55.904489 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 02:56:55.904791 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":305,"threads_started":1}
I20260808 02:56:55.906567 1403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:56:55.990803 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1219
I20260808 02:56:56.019091 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1422
Aug 08 02:56:56 dist-test-slave-6ntk krb5kdc[836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:56:56.044438 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:56.020278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.0.205.131:32849)
0808 02:56:56.020966 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:56.020973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:56.021004 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:56:56.021068 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:56.021074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:56.021292 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:56.021449 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:56.022787 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:56.023863 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:56.025118 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:56.025683 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:56.025781 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:56.038406 (+ 12625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:56.038421 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:56.038437 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:56.038478 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:56.041468 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:56.042298 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:56.042305 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:56.042307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:56.042444 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:56.042794 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:56.042797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:56.042799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:56.043244 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:56.043560 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:56.043828 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 02:56:56.044077 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":375,"threads_started":1}
I20260808 02:56:56.045830 1613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:56:56.131907 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1422
I20260808 02:56:56.157739 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 859
I20260808 02:56:56.159032 947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:56:56.270170 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 859
2026-08-08T02:56:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4587 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:56:56 dist-test-slave-6ntk krb5kdc[1649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:56:56 dist-test-slave-6ntk krb5kdc[1649](info): set up 2 sockets
Aug 08 02:56:56 dist-test-slave-6ntk krb5kdc[1649](info): commencing operation
krb5kdc: starting...
W20260808 02:56:57.568666 822 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.005s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:56:57 dist-test-slave-6ntk krb5kdc[1649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:56:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:56:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:56:57.769508 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:35653
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:35653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:56:58.102914 1670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:56:58.103281 1670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:56:58.103459 1670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:56:58.110914 1670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:56:58.111073 1670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:56:58.111195 1670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:56:58.111292 1670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:56:58.111378 1670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:56:58.121260 1670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:35653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:35653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:56:58.122642 1670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:56:58.123994 1670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:56:58.135968 1675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:58.135896 1677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:58.135936 1680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:56:58.137130 1670 server_base.cc:1034] running on GCE node
I20260808 02:56:58.137970 1670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:56:58.139415 1670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:56:58.140659 1670 hybrid_clock.cc:648] HybridClock initialized: now 1786157818140619 us; error 53 us; skew 500 ppm
I20260808 02:56:58.141012 1670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:56:58 dist-test-slave-6ntk krb5kdc[1649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:56:58.144127 1670 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:56:58.145854 1670 webserver.cc:460] Webserver started at http://127.0.205.190:45823/ using document root <none> and password file <none>
I20260808 02:56:58.146651 1670 fs_manager.cc:359] Metadata directory not provided
I20260808 02:56:58.146801 1670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:56:58.147128 1670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:56:58.150182 1670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "7e614a7132b0458793a9c76d096ba302"
format_stamp: "Formatted at 2026-08-08 02:56:58 on dist-test-slave-6ntk"
server_key: "06ebf57beac40d8252bff617e1c251ad"
server_key_iv: "66182c9c4c05553eb552bb86fe791ef7"
server_key_version: "encryptionkey@0"
I20260808 02:56:58.150959 1670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "7e614a7132b0458793a9c76d096ba302"
format_stamp: "Formatted at 2026-08-08 02:56:58 on dist-test-slave-6ntk"
server_key: "06ebf57beac40d8252bff617e1c251ad"
server_key_iv: "66182c9c4c05553eb552bb86fe791ef7"
server_key_version: "encryptionkey@0"
I20260808 02:56:58.156229 1670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 02:56:58.159693 1692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:58.160997 1670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:56:58.161188 1670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "7e614a7132b0458793a9c76d096ba302"
format_stamp: "Formatted at 2026-08-08 02:56:58 on dist-test-slave-6ntk"
server_key: "06ebf57beac40d8252bff617e1c251ad"
server_key_iv: "66182c9c4c05553eb552bb86fe791ef7"
server_key_version: "encryptionkey@0"
I20260808 02:56:58.161402 1670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:56:58.178304 1670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:56:58.182560 1670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:56:58.182873 1670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:56:58.208173 1670 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:35653
I20260808 02:56:58.208174 1775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:35653 every 8 connection(s)
I20260808 02:56:58.209810 1670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:56:58.214038 1778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:56:58.219064 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1670
I20260808 02:56:58.219332 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:56:58.219993 822 external_mini_cluster.cc:1428] Setting key 2cc1df51c0ee27a87895dc3dcbe87b87
I20260808 02:56:58.225543 1778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302: Bootstrap starting.
I20260808 02:56:58.229508 1778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302: Neither blocks nor log segments found. Creating new log.
I20260808 02:56:58.230839 1778 log.cc:824] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302: Log is configured to *not* fsync() on all Append() calls
I20260808 02:56:58.234102 1778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302: No bootstrap required, opened a new log
Aug 08 02:56:58 dist-test-slave-6ntk krb5kdc[1649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:56:58.243993 1778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e614a7132b0458793a9c76d096ba302" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35653 } }
I20260808 02:56:58.244282 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:58.222079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42260 (local address 127.0.205.190:35653)
0808 02:56:58.222960 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:58.222971 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:58.223014 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:56:58.224880 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:58.224910 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:58.225314 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:58.225655 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:58.227562 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:58.228858 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:58.230739 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:58.231634 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:58.231838 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:58.236478 (+ 4640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:58.236486 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:58.236502 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:58.236541 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:58.239856 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:58.240713 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:58.240720 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:58.240727 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:58.240905 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:58.241312 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:58.241315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:58.241317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:58.241990 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:58.242226 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:58.242789 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 02:56:58.243205 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":409,"threads_started":1}
I20260808 02:56:58.244412 1778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:56:58.244520 1778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e614a7132b0458793a9c76d096ba302, State: Initialized, Role: FOLLOWER
I20260808 02:56:58.245162 1778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e614a7132b0458793a9c76d096ba302" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35653 } }
I20260808 02:56:58.245419 1778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:56:58.245568 1778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:56:58.245766 1778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:56:58.248194 1778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e614a7132b0458793a9c76d096ba302" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35653 } }
I20260808 02:56:58.248718 1778 leader_election.cc:302] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e614a7132b0458793a9c76d096ba302; no voters:
I20260808 02:56:58.249300 1778 leader_election.cc:288] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:56:58.249437 1789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:56:58.250715 1789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [term 1 LEADER]: Becoming Leader. State: Replica: 7e614a7132b0458793a9c76d096ba302, State: Running, Role: LEADER
I20260808 02:56:58.251374 1789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e614a7132b0458793a9c76d096ba302" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35653 } }
I20260808 02:56:58.251526 1778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:56:58.254627 1791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e614a7132b0458793a9c76d096ba302. Latest consensus state: current_term: 1 leader_uuid: "7e614a7132b0458793a9c76d096ba302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e614a7132b0458793a9c76d096ba302" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35653 } } }
I20260808 02:56:58.255087 1791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [sys.catalog]: This master's current role is: LEADER
I20260808 02:56:58.254989 1790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e614a7132b0458793a9c76d096ba302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e614a7132b0458793a9c76d096ba302" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35653 } } }
I20260808 02:56:58.255275 1790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302 [sys.catalog]: This master's current role is: LEADER
I20260808 02:56:58.255983 1800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:56:58.261839 1800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:56:58.271631 1800 catalog_manager.cc:1347] Generated new cluster ID: 1b812f4c163f4e568910aa9e84750f72
I20260808 02:56:58.271756 1800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:56:58.285073 1800 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:56:58.286530 1800 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:56:58.292665 1800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e614a7132b0458793a9c76d096ba302: Generated new TSK 0
I20260808 02:56:58.293570 1800 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:56:58.353780 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35653
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:56:58.700765 1817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:56:58.701083 1817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:56:58.701267 1817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:56:58.708812 1817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:56:58.708925 1817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:56:58.709093 1817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:56:58.719185 1817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35653
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:56:58.720568 1817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:56:58.721963 1817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:56:58.735935 1823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:58.735962 1826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:58.735873 1824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:56:58.736781 1817 server_base.cc:1034] running on GCE node
I20260808 02:56:58.737350 1817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:56:58.738552 1817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:56:58.739815 1817 hybrid_clock.cc:648] HybridClock initialized: now 1786157818739737 us; error 57 us; skew 500 ppm
I20260808 02:56:58.740147 1817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:56:58 dist-test-slave-6ntk krb5kdc[1649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:56:58.743480 1817 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:56:58.745316 1817 webserver.cc:460] Webserver started at http://127.0.205.129:35989/ using document root <none> and password file <none>
I20260808 02:56:58.746156 1817 fs_manager.cc:359] Metadata directory not provided
I20260808 02:56:58.746306 1817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:56:58.746690 1817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:56:58.750272 1817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "2341d2256ce5454cb6bcb204e3944ff9"
format_stamp: "Formatted at 2026-08-08 02:56:58 on dist-test-slave-6ntk"
server_key: "72347d0f9084916d4a357b436c578c3f"
server_key_iv: "746dfa9f7a161e8645136e8b5df210f2"
server_key_version: "encryptionkey@0"
I20260808 02:56:58.751122 1817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "2341d2256ce5454cb6bcb204e3944ff9"
format_stamp: "Formatted at 2026-08-08 02:56:58 on dist-test-slave-6ntk"
server_key: "72347d0f9084916d4a357b436c578c3f"
server_key_iv: "746dfa9f7a161e8645136e8b5df210f2"
server_key_version: "encryptionkey@0"
I20260808 02:56:58.757233 1817 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260808 02:56:58.761003 1841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:58.762552 1817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:56:58.762746 1817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "2341d2256ce5454cb6bcb204e3944ff9"
format_stamp: "Formatted at 2026-08-08 02:56:58 on dist-test-slave-6ntk"
server_key: "72347d0f9084916d4a357b436c578c3f"
server_key_iv: "746dfa9f7a161e8645136e8b5df210f2"
server_key_version: "encryptionkey@0"
I20260808 02:56:58.762972 1817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:56:58.788272 1817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:56:58.792932 1817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:56:58.793265 1817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:56:58.794440 1817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:56:58.795831 1817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:56:58.795944 1817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:58.796093 1817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:56:58.796166 1817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:58.830037 1817 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:44063
I20260808 02:56:58.830077 2007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:44063 every 8 connection(s)
I20260808 02:56:58.831362 1817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:56:58.834772 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1817
I20260808 02:56:58.835032 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:56:58.835870 822 external_mini_cluster.cc:1428] Setting key 581e5725baaebb47601f5169467da615
Aug 08 02:56:58 dist-test-slave-6ntk krb5kdc[1649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:56:58.852453 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:58.834015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:35675 (local address 127.0.205.190:35653)
0808 02:56:58.834514 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:58.834519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:58.835612 (+ 1093us) server_negotiation.cc:407] Connection header received
0808 02:56:58.837800 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:58.837805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:58.837926 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:58.838055 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:58.839621 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:58.840345 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:58.842013 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:58.842446 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:58.842591 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:58.846533 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:58.846538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:58.846542 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:58.846577 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:58.849185 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:58.850009 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:58.850014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:58.850016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:58.850137 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:58.850593 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:58.850605 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:58.850607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:58.850877 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:58.851041 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:58.851828 (+ 787us) server_negotiation.cc:299] Negotiation successful
0808 02:56:58.852058 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":266,"threads_started":1}
I20260808 02:56:58.853541 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:58.834586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35653 (local address 127.0.205.129:35675)
0808 02:56:58.835448 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0808 02:56:58.835487 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:56:58.837483 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:56:58.838319 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:56:58.838328 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:56:58.838845 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:56:58.839438 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:58.839456 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:58.840523 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:58.840528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:58.841784 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:58.841796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:58.842702 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:58.842710 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:58.842963 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:58.843712 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:56:58.843735 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:56:58.846284 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:56:58.849344 (+ 3060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:58.849354 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:58.849374 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:58.849833 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:58.850255 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:58.850260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:58.850264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:58.850468 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:58.851022 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:56:58.851031 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:56:58.851430 (+ 399us) client_negotiation.cc:769] Sending connection context
0808 02:56:58.851705 (+ 275us) client_negotiation.cc:240] Negotiation successful
0808 02:56:58.852323 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":345,"threads_started":1}
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
I20260808 02:56:58.855293 2008 heartbeater.cc:347] Connected to a master server at 127.0.205.190:35653
I20260808 02:56:58.855695 2008 heartbeater.cc:464] Registering TS with master...
I20260808 02:56:58.856477 2008 heartbeater.cc:511] Master 127.0.205.190:35653 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
I20260808 02:56:58.859028 1720 ts_manager.cc:194] Registered new tserver with Master: 2341d2256ce5454cb6bcb204e3944ff9 (127.0.205.129:44063)
I20260808 02:56:58.861238 1720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:35675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:56:58.884773 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35653
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:56:59.232892 2018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:56:59.233239 2018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:56:59.233438 2018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:56:59.240916 2018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:56:59.241066 2018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:56:59.241273 2018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:56:59.251593 2018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35653
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:56:59.253015 2018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:56:59.254606 2018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:56:59.268682 2023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:59.268738 2024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:59.268853 2026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:56:59.269644 2018 server_base.cc:1034] running on GCE node
I20260808 02:56:59.270452 2018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:56:59.271461 2018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:56:59.272701 2018 hybrid_clock.cc:648] HybridClock initialized: now 1786157819272648 us; error 46 us; skew 500 ppm
I20260808 02:56:59.273049 2018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:56:59 dist-test-slave-6ntk krb5kdc[1649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:56:59.276295 2018 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:56:59.277956 2018 webserver.cc:460] Webserver started at http://127.0.205.130:44851/ using document root <none> and password file <none>
I20260808 02:56:59.278695 2018 fs_manager.cc:359] Metadata directory not provided
I20260808 02:56:59.278853 2018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:56:59.279182 2018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:56:59.282256 2018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "6da8bbc41e9f47d3b551082000a95f37"
format_stamp: "Formatted at 2026-08-08 02:56:59 on dist-test-slave-6ntk"
server_key: "37426e077c0961f7ab9ce7b23bb84735"
server_key_iv: "d714a345dc879eab6e9c760a2298b980"
server_key_version: "encryptionkey@0"
I20260808 02:56:59.283106 2018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "6da8bbc41e9f47d3b551082000a95f37"
format_stamp: "Formatted at 2026-08-08 02:56:59 on dist-test-slave-6ntk"
server_key: "37426e077c0961f7ab9ce7b23bb84735"
server_key_iv: "d714a345dc879eab6e9c760a2298b980"
server_key_version: "encryptionkey@0"
I20260808 02:56:59.288507 2018 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 02:56:59.291993 2034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:59.293676 2018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:56:59.293967 2018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "6da8bbc41e9f47d3b551082000a95f37"
format_stamp: "Formatted at 2026-08-08 02:56:59 on dist-test-slave-6ntk"
server_key: "37426e077c0961f7ab9ce7b23bb84735"
server_key_iv: "d714a345dc879eab6e9c760a2298b980"
server_key_version: "encryptionkey@0"
I20260808 02:56:59.294201 2018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:56:59.309190 2018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:56:59.313434 2018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:56:59.313760 2018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:56:59.314791 2018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:56:59.316185 2018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:56:59.316304 2018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:59.316468 2018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:56:59.316550 2018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:59.351157 2018 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:39265
I20260808 02:56:59.351364 2197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:39265 every 8 connection(s)
I20260808 02:56:59.352669 2018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:56:59.355472 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 2018
I20260808 02:56:59.355697 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:56:59.356204 822 external_mini_cluster.cc:1428] Setting key 1d68442d56234bdd81b6cd9811926d1f
Aug 08 02:56:59 dist-test-slave-6ntk krb5kdc[1649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
I20260808 02:56:59.377493 2200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.355553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:53313 (local address 127.0.205.190:35653)
0808 02:56:59.356045 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:59.356052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:59.357815 (+ 1763us) server_negotiation.cc:407] Connection header received
0808 02:56:59.360593 (+ 2778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:59.360597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:59.360664 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:59.360759 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:59.363057 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.363763 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.365844 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.366671 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.366816 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.370909 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:59.370914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:59.370917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:59.370952 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:59.373702 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:59.374846 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:59.374850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:59.374852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:59.374998 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:59.375545 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:59.375548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:59.375550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:59.375826 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:59.375958 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:59.376975 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0808 02:56:59.377152 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":230,"threads_started":1}
I20260808 02:56:59.378983 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.356200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35653 (local address 127.0.205.130:53313)
0808 02:56:59.357647 (+ 1447us) negotiation.cc:107] Waiting for socket to connect
0808 02:56:59.357691 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:56:59.360283 (+ 2592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:56:59.361058 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:56:59.361073 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:56:59.362005 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:56:59.362849 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:59.362877 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.363934 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.363943 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:59.365695 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:59.365707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.366977 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.366985 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:59.367192 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.367921 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:56:59.367964 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:56:59.370641 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:56:59.373937 (+ 3296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:59.373952 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:59.373974 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:59.374709 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:59.375121 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:59.375128 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:59.375133 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:59.375421 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:59.375978 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:56:59.375986 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:56:59.376500 (+ 514us) client_negotiation.cc:769] Sending connection context
0808 02:56:59.376883 (+ 383us) client_negotiation.cc:240] Negotiation successful
0808 02:56:59.377503 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1227,"thread_start_us":320,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
I20260808 02:56:59.380587 2198 heartbeater.cc:347] Connected to a master server at 127.0.205.190:35653
I20260808 02:56:59.380944 2198 heartbeater.cc:464] Registering TS with master...
I20260808 02:56:59.381938 2198 heartbeater.cc:511] Master 127.0.205.190:35653 requested a full tablet report, sending...
I20260808 02:56:59.383569 1720 ts_manager.cc:194] Registered new tserver with Master: 6da8bbc41e9f47d3b551082000a95f37 (127.0.205.130:39265)
I20260808 02:56:59.384402 1720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:53313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:56:59.406783 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35653
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:56:59.746573 2209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:56:59.746917 2209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:56:59.747100 2209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:56:59.754829 2209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:56:59.754992 2209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:56:59.755168 2209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:56:59.765684 2209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35653
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:56:59.767320 2209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:56:59.768774 2209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:56:59.782567 2214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:59.782490 2219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:56:59.782588 2215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:56:59.783623 2209 server_base.cc:1034] running on GCE node
I20260808 02:56:59.784286 2209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:56:59.785359 2209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:56:59.786639 2209 hybrid_clock.cc:648] HybridClock initialized: now 1786157819786587 us; error 61 us; skew 500 ppm
I20260808 02:56:59.787014 2209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:56:59 dist-test-slave-6ntk krb5kdc[1649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:56:59.790638 2209 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:56:59.792325 2209 webserver.cc:460] Webserver started at http://127.0.205.131:41241/ using document root <none> and password file <none>
I20260808 02:56:59.793066 2209 fs_manager.cc:359] Metadata directory not provided
I20260808 02:56:59.793205 2209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:56:59.793557 2209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:56:59.796731 2209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "be32b1a8ef9b45bab83c0fa097aa08e1"
format_stamp: "Formatted at 2026-08-08 02:56:59 on dist-test-slave-6ntk"
server_key: "75206e1582f6575df6938a619e52970e"
server_key_iv: "25e305c8999bd6523cda9fe6f4f0e400"
server_key_version: "encryptionkey@0"
I20260808 02:56:59.797617 2209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "be32b1a8ef9b45bab83c0fa097aa08e1"
format_stamp: "Formatted at 2026-08-08 02:56:59 on dist-test-slave-6ntk"
server_key: "75206e1582f6575df6938a619e52970e"
server_key_iv: "25e305c8999bd6523cda9fe6f4f0e400"
server_key_version: "encryptionkey@0"
I20260808 02:56:59.803355 2209 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 02:56:59.807016 2228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:59.808414 2209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 02:56:59.808611 2209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "be32b1a8ef9b45bab83c0fa097aa08e1"
format_stamp: "Formatted at 2026-08-08 02:56:59 on dist-test-slave-6ntk"
server_key: "75206e1582f6575df6938a619e52970e"
server_key_iv: "25e305c8999bd6523cda9fe6f4f0e400"
server_key_version: "encryptionkey@0"
I20260808 02:56:59.808842 2209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:56:59.821218 2209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:56:59.825438 2209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:56:59.825749 2209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:56:59.826851 2209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:56:59.828228 2209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:56:59.828338 2209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:59.828500 2209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:56:59.828575 2209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:56:59.863157 2209 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:35143
I20260808 02:56:59.863173 2404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:35143 every 8 connection(s)
I20260808 02:56:59.864545 2209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:56:59.864974 2008 heartbeater.cc:503] Master 127.0.205.190:35653 was elected leader, sending a full tablet report...
I20260808 02:56:59.867497 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 2209
I20260808 02:56:59.867722 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:56:59.868270 822 external_mini_cluster.cc:1428] Setting key 5f0a443fa8dc7d77dcb9a04bb478bd24
Aug 08 02:56:59 dist-test-slave-6ntk krb5kdc[1649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:56:59.886768 2200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.868030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:45991 (local address 127.0.205.190:35653)
0808 02:56:59.868235 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:59.868241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:59.869540 (+ 1299us) server_negotiation.cc:407] Connection header received
0808 02:56:59.871636 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:59.871640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:59.871714 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:59.871793 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:59.873465 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.874336 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.875963 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.876371 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.876478 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.880178 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:59.880183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:59.880186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:59.880220 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:59.883496 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:59.884266 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:59.884273 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:59.884276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:59.884417 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:59.884861 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:59.884868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:59.884871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:59.885163 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:59.885338 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:59.886160 (+ 822us) server_negotiation.cc:299] Negotiation successful
0808 02:56:59.886406 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:56:59.887907 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.868508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35653 (local address 127.0.205.131:45991)
0808 02:56:59.869391 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0808 02:56:59.869442 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:56:59.871374 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:56:59.872059 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:56:59.872068 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:56:59.872651 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:56:59.873282 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:59.873300 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.874504 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.874509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:59.875779 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:59.875797 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.876591 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.876596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:59.876798 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.877484 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:56:59.877506 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:56:59.879969 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:56:59.883647 (+ 3678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:59.883657 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:59.883676 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:59.884136 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:59.884536 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:56:59.884541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:56:59.884545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:56:59.884743 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:56:59.885307 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:56:59.885315 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:56:59.885684 (+ 369us) client_negotiation.cc:769] Sending connection context
0808 02:56:59.886049 (+ 365us) client_negotiation.cc:240] Negotiation successful
0808 02:56:59.886656 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":408,"threads_started":1}
I20260808 02:56:59.889398 2407 heartbeater.cc:347] Connected to a master server at 127.0.205.190:35653
I20260808 02:56:59.889794 2407 heartbeater.cc:464] Registering TS with master...
I20260808 02:56:59.890939 2407 heartbeater.cc:511] Master 127.0.205.190:35653 requested a full tablet report, sending...
I20260808 02:56:59.892565 1720 ts_manager.cc:194] Registered new tserver with Master: be32b1a8ef9b45bab83c0fa097aa08e1 (127.0.205.131:35143)
I20260808 02:56:59.893435 1720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:45991
I20260808 02:56:59.896242 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:56:59.917021 2200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.904037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42262 (local address 127.0.205.190:35653)
0808 02:56:59.904239 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:59.904243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:59.904863 (+ 620us) server_negotiation.cc:407] Connection header received
0808 02:56:59.905029 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:59.905033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:59.905108 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:59.905202 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:56:59.906966 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.907707 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.908999 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.909403 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.909498 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.911895 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:56:59.911900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:56:59.911903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:56:59.911936 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:56:59.914746 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:59.915341 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:59.915345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:59.915347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:59.915474 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:56:59.915797 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:56:59.915800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:56:59.915802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:56:59.916119 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:56:59.916317 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:59.916598 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 02:56:59.916757 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:56:59.925649 1720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:56:59.933250 1720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:56:59.979838 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.949973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:35143 (local address 127.0.0.1:47180)
0808 02:56:59.965339 (+ 15366us) negotiation.cc:107] Waiting for socket to connect
0808 02:56:59.965418 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 02:56:59.965829 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:56:59.968863 (+ 3034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:56:59.968875 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:56:59.968918 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:56:59.969536 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:59.969553 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.972278 (+ 2725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.972286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:59.975696 (+ 3410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:59.975716 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.978429 (+ 2713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.978438 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:59.978714 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.978826 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 02:56:59.979028 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 02:56:59.979286 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14981,"thread_start_us":16142,"threads_started":1}
I20260808 02:56:59.981036 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.950327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47180 (local address 127.0.205.131:35143)
0808 02:56:59.967598 (+ 17271us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:59.967611 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:59.967656 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:56:59.967777 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:59.967787 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:59.968154 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:59.968453 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:56:59.969828 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.972052 (+ 2224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.975929 (+ 3877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.977727 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.977928 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.978244 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:59.980360 (+ 2116us) server_negotiation.cc:299] Negotiation successful
0808 02:56:59.980539 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17015,"thread_start_us":16819,"threads_started":1}
I20260808 02:56:59.985423 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.950414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:39265 (local address 127.0.0.1:35086)
0808 02:56:59.968048 (+ 17634us) negotiation.cc:107] Waiting for socket to connect
0808 02:56:59.968073 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:56:59.968263 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:56:59.969850 (+ 1587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:56:59.969858 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:56:59.969918 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:56:59.971965 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:59.971985 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.974511 (+ 2526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.974520 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:59.979042 (+ 4522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:59.979064 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.982883 (+ 3819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.982891 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:59.983172 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.983219 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 02:56:59.983327 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 02:56:59.983481 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17470,"thread_start_us":17294,"threads_started":1}
I20260808 02:56:59.987208 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.950470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35086 (local address 127.0.205.130:39265)
0808 02:56:59.968816 (+ 18346us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:59.968824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:59.968851 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:56:59.968949 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:59.968959 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:59.969215 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:59.969522 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:56:59.972233 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.974301 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.979326 (+ 5025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.982515 (+ 3189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.986344 (+ 3829us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.986578 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:59.986658 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 02:56:59.986843 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18177,"thread_start_us":18078,"threads_started":1}
I20260808 02:56:59.986904 2299 tablet_service.cc:1467] Processing CreateTablet for tablet 83fc9276fa0b4619893a71a74fbc4661 (DEFAULT_TABLE table=test-table [id=513f6dc9672c49fda2408496a511ac9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:56:59.989636 2299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83fc9276fa0b4619893a71a74fbc4661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:56:59.990695 2101 tablet_service.cc:1467] Processing CreateTablet for tablet 83fc9276fa0b4619893a71a74fbc4661 (DEFAULT_TABLE table=test-table [id=513f6dc9672c49fda2408496a511ac9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:56:59.991639 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.968876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44063 (local address 127.0.0.1:51470)
0808 02:56:59.982961 (+ 14085us) negotiation.cc:107] Waiting for socket to connect
0808 02:56:59.982992 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 02:56:59.983133 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:56:59.985129 (+ 1996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:56:59.985137 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:56:59.985176 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:56:59.985452 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:59.985467 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.987119 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.987126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:59.988724 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:56:59.988739 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.990770 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.990776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:56:59.990957 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.990995 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 02:56:59.991097 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 02:56:59.991240 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13932,"thread_start_us":16943,"threads_started":1}
I20260808 02:56:59.991720 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:56:59.968390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.0.205.129:44063)
0808 02:56:59.983854 (+ 15464us) server_negotiation.cc:206] Beginning negotiation
0808 02:56:59.983863 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:56:59.983893 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:56:59.984019 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:56:59.984030 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:56:59.984377 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:56:59.984754 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:56:59.985668 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.986968 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.988930 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:56:59.990551 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:56:59.990734 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:56:59.990984 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 02:56:59.991186 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 02:56:59.991363 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15269,"thread_start_us":15075,"threads_started":1}
I20260808 02:56:59.992184 2101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83fc9276fa0b4619893a71a74fbc4661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:56:59.995949 1904 tablet_service.cc:1467] Processing CreateTablet for tablet 83fc9276fa0b4619893a71a74fbc4661 (DEFAULT_TABLE table=test-table [id=513f6dc9672c49fda2408496a511ac9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:56:59.997429 1904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83fc9276fa0b4619893a71a74fbc4661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:00.006549 2430 tablet_bootstrap.cc:492] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1: Bootstrap starting.
I20260808 02:57:00.008759 2431 tablet_bootstrap.cc:492] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37: Bootstrap starting.
I20260808 02:57:00.011047 2430 tablet_bootstrap.cc:654] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:00.012563 2430 log.cc:824] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:00.012814 2431 tablet_bootstrap.cc:654] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:00.012866 2433 tablet_bootstrap.cc:492] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9: Bootstrap starting.
I20260808 02:57:00.014298 2431 log.cc:824] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:00.016479 2430 tablet_bootstrap.cc:492] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1: No bootstrap required, opened a new log
I20260808 02:57:00.016794 2430 ts_tablet_manager.cc:1397] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260808 02:57:00.016832 2433 tablet_bootstrap.cc:654] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:00.017843 2431 tablet_bootstrap.cc:492] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37: No bootstrap required, opened a new log
I20260808 02:57:00.018075 2433 log.cc:824] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:00.018146 2431 ts_tablet_manager.cc:1397] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260808 02:57:00.024183 2433 tablet_bootstrap.cc:492] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9: No bootstrap required, opened a new log
I20260808 02:57:00.024507 2433 ts_tablet_manager.cc:1397] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260808 02:57:00.029208 2431 raft_consensus.cc:348] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } }
I20260808 02:57:00.029206 2430 raft_consensus.cc:348] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } }
I20260808 02:57:00.029717 2431 raft_consensus.cc:374] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:00.029717 2430 raft_consensus.cc:374] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:00.029829 2430 raft_consensus.cc:729] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be32b1a8ef9b45bab83c0fa097aa08e1, State: Initialized, Role: FOLLOWER
I20260808 02:57:00.029861 2431 raft_consensus.cc:729] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6da8bbc41e9f47d3b551082000a95f37, State: Initialized, Role: FOLLOWER
I20260808 02:57:00.030647 2431 consensus_queue.cc:260] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } }
I20260808 02:57:00.030709 2430 consensus_queue.cc:260] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } }
I20260808 02:57:00.035059 2407 heartbeater.cc:503] Master 127.0.205.190:35653 was elected leader, sending a full tablet report...
I20260808 02:57:00.035099 2198 heartbeater.cc:503] Master 127.0.205.190:35653 was elected leader, sending a full tablet report...
I20260808 02:57:00.036103 2433 raft_consensus.cc:348] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } }
I20260808 02:57:00.036752 2433 raft_consensus.cc:374] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:00.036911 2433 raft_consensus.cc:729] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2341d2256ce5454cb6bcb204e3944ff9, State: Initialized, Role: FOLLOWER
I20260808 02:57:00.037972 2430 ts_tablet_manager.cc:1428] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1: Time spent starting tablet: real 0.021s user 0.018s sys 0.003s
I20260808 02:57:00.037819 2433 consensus_queue.cc:260] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } }
I20260808 02:57:00.038882 2431 ts_tablet_manager.cc:1428] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37: Time spent starting tablet: real 0.021s user 0.013s sys 0.004s
I20260808 02:57:00.039307 2438 raft_consensus.cc:482] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:00.039598 2438 raft_consensus.cc:504] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } }
I20260808 02:57:00.040822 2433 ts_tablet_manager.cc:1428] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260808 02:57:00.043309 2439 raft_consensus.cc:482] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:00.043767 2439 raft_consensus.cc:504] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } }
I20260808 02:57:00.045642 2438 leader_election.cc:288] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6da8bbc41e9f47d3b551082000a95f37 (127.0.205.130:39265), 2341d2256ce5454cb6bcb204e3944ff9 (127.0.205.129:44063)
I20260808 02:57:00.052222 2439 leader_election.cc:288] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be32b1a8ef9b45bab83c0fa097aa08e1 (127.0.205.131:35143), 2341d2256ce5454cb6bcb204e3944ff9 (127.0.205.129:44063)
I20260808 02:57:00.053413 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.042542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:39265 (local address 127.0.205.131:56571)
0808 02:57:00.042755 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:00.042779 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:00.042889 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:00.043884 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:00.043889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:00.043916 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:00.044136 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:00.044153 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.046088 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.046094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:00.048659 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:00.048680 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.052345 (+ 3665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.052358 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:00.052642 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.052703 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 02:57:00.052803 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 02:57:00.052998 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 02:57:00.054567 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.043100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:56571 (local address 127.0.205.130:39265)
0808 02:57:00.043357 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:00.043366 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:00.043387 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:57:00.043466 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:00.043470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:00.043551 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:00.043656 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:00.044398 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.045830 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.050117 (+ 4287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.051938 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.053893 (+ 1955us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.053974 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:00.054028 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 02:57:00.054195 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:57:00.063992 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.043805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:47689 (local address 127.0.205.129:44063)
0808 02:57:00.043996 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:00.044002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:00.053805 (+ 9803us) server_negotiation.cc:407] Connection header received
0808 02:57:00.055185 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:00.055193 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:00.055307 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:00.055382 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:00.056239 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.057674 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.060514 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.062395 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.062576 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.062690 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:00.063431 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 02:57:00.063627 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:57:00.065167 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.043103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44063 (local address 127.0.205.131:47689)
0808 02:57:00.053693 (+ 10590us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:00.053717 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:00.054851 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:00.055665 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:00.055672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:00.055706 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:00.056051 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:00.056068 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.057933 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.057941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:00.060283 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:00.060302 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.062882 (+ 2580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.062891 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:00.063189 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.063244 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 02:57:00.064484 (+ 1240us) client_negotiation.cc:240] Negotiation successful
0808 02:57:00.064690 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10461,"threads_started":1}
I20260808 02:57:00.068102 1938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fc9276fa0b4619893a71a74fbc4661" candidate_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2341d2256ce5454cb6bcb204e3944ff9" is_pre_election: true
I20260808 02:57:00.069104 1938 raft_consensus.cc:2454] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be32b1a8ef9b45bab83c0fa097aa08e1 in term 0.
I20260808 02:57:00.070361 2232 leader_election.cc:302] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2341d2256ce5454cb6bcb204e3944ff9, be32b1a8ef9b45bab83c0fa097aa08e1; no voters:
I20260808 02:57:00.070307 2126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fc9276fa0b4619893a71a74fbc4661" candidate_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6da8bbc41e9f47d3b551082000a95f37" is_pre_election: true
I20260808 02:57:00.070919 2126 raft_consensus.cc:2454] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be32b1a8ef9b45bab83c0fa097aa08e1 in term 0.
I20260808 02:57:00.071280 2438 raft_consensus.cc:2781] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:00.071480 2438 raft_consensus.cc:482] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:00.071636 2438 raft_consensus.cc:3037] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:00.075044 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.046179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:41721 (local address 127.0.205.131:35143)
0808 02:57:00.046387 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:00.046395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:00.066939 (+ 20544us) server_negotiation.cc:407] Connection header received
0808 02:57:00.067258 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:00.067262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:00.067365 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:00.067435 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:00.069016 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.070166 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.072900 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.073996 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.074186 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.074276 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:00.074618 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 02:57:00.074761 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:57:00.075258 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.046335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:35143 (local address 127.0.205.130:41721)
0808 02:57:00.066029 (+ 19694us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:00.066057 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:00.067083 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:00.067666 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:00.067670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:00.067701 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:00.068196 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:00.068212 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.070355 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.070360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:00.072729 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:00.072742 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.074252 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.074258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:00.074449 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.074474 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:57:00.074596 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 02:57:00.074824 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19440,"mutex_wait_us":76,"thread_start_us":19184,"threads_started":1}
I20260808 02:57:00.075498 2438 raft_consensus.cc:504] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } }
I20260808 02:57:00.076273 2438 leader_election.cc:288] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [CANDIDATE]: Term 1 election: Requested vote from peers 6da8bbc41e9f47d3b551082000a95f37 (127.0.205.130:39265), 2341d2256ce5454cb6bcb204e3944ff9 (127.0.205.129:44063)
I20260808 02:57:00.076429 2333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fc9276fa0b4619893a71a74fbc4661" candidate_uuid: "6da8bbc41e9f47d3b551082000a95f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" is_pre_election: true
I20260808 02:57:00.076749 2126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fc9276fa0b4619893a71a74fbc4661" candidate_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6da8bbc41e9f47d3b551082000a95f37"
I20260808 02:57:00.076967 2126 raft_consensus.cc:3037] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:00.077179 2333 raft_consensus.cc:2379] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6da8bbc41e9f47d3b551082000a95f37 in current term 1: Already voted for candidate be32b1a8ef9b45bab83c0fa097aa08e1 in this term.
I20260808 02:57:00.077214 1938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fc9276fa0b4619893a71a74fbc4661" candidate_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2341d2256ce5454cb6bcb204e3944ff9"
I20260808 02:57:00.077471 1938 raft_consensus.cc:3037] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:00.079557 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.046957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:56835 (local address 127.0.205.129:44063)
0808 02:57:00.064432 (+ 17475us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:00.064441 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:00.070584 (+ 6143us) server_negotiation.cc:407] Connection header received
0808 02:57:00.071599 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:00.071605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:00.071692 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:00.071767 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:00.072834 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.074006 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.076695 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.078206 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.078366 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.078439 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:00.079079 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 02:57:00.079226 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17325,"thread_start_us":25061,"threads_started":1}
I20260808 02:57:00.079617 2446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.046513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44063 (local address 127.0.205.130:56835)
0808 02:57:00.068933 (+ 22420us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:00.068955 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:00.071359 (+ 2404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:00.072070 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:00.072078 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:00.072109 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:00.072631 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:00.072650 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.074179 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.074186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:00.076482 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:00.076495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.078652 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.078658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:00.078878 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.078912 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:57:00.079038 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 02:57:00.079192 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22276,"thread_start_us":22125,"threads_started":1}
I20260808 02:57:00.080466 1937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83fc9276fa0b4619893a71a74fbc4661" candidate_uuid: "6da8bbc41e9f47d3b551082000a95f37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2341d2256ce5454cb6bcb204e3944ff9" is_pre_election: true
I20260808 02:57:00.081274 2126 raft_consensus.cc:2454] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be32b1a8ef9b45bab83c0fa097aa08e1 in term 1.
I20260808 02:57:00.081676 2232 leader_election.cc:302] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6da8bbc41e9f47d3b551082000a95f37, be32b1a8ef9b45bab83c0fa097aa08e1; no voters:
I20260808 02:57:00.081707 2039 leader_election.cc:302] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6da8bbc41e9f47d3b551082000a95f37; no voters: 2341d2256ce5454cb6bcb204e3944ff9, be32b1a8ef9b45bab83c0fa097aa08e1
I20260808 02:57:00.082091 2438 raft_consensus.cc:2781] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:00.082044 1938 raft_consensus.cc:2454] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be32b1a8ef9b45bab83c0fa097aa08e1 in term 1.
I20260808 02:57:00.082278 2439 raft_consensus.cc:2726] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 02:57:00.082949 2438 raft_consensus.cc:686] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [term 1 LEADER]: Becoming Leader. State: Replica: be32b1a8ef9b45bab83c0fa097aa08e1, State: Running, Role: LEADER
I20260808 02:57:00.083556 2438 consensus_queue.cc:237] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } }
W20260808 02:57:00.084038 2010 tablet.cc:2369] T 83fc9276fa0b4619893a71a74fbc4661 P 2341d2256ce5454cb6bcb204e3944ff9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:57:00.089107 1720 catalog_manager.cc:5495] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to be32b1a8ef9b45bab83c0fa097aa08e1 (127.0.205.131). New cstate: current_term: 1 leader_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be32b1a8ef9b45bab83c0fa097aa08e1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6da8bbc41e9f47d3b551082000a95f37" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2341d2256ce5454cb6bcb204e3944ff9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44063 } health_report { overall_health: UNKNOWN } } }
W20260808 02:57:00.105054 2199 tablet.cc:2369] T 83fc9276fa0b4619893a71a74fbc4661 P 6da8bbc41e9f47d3b551082000a95f37: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:57:00.116843 2408 tablet.cc:2369] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 02:57:00 dist-test-slave-6ntk krb5kdc[1649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:00.129992 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.114112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51476 (local address 127.0.205.129:44063)
0808 02:57:00.114343 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:00.114352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:00.115169 (+ 817us) server_negotiation.cc:407] Connection header received
0808 02:57:00.115325 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:00.115329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:00.115414 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:00.115503 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:00.117397 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.118289 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.119558 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.119984 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.120080 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.124395 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:00.124417 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:00.124435 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:00.124482 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:00.127421 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.128085 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.128092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.128095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.128229 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.128561 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.128564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.128566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.128949 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:00.129181 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:00.129420 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 02:57:00.129587 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 02:57:00 dist-test-slave-6ntk krb5kdc[1649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157820, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:57:00 dist-test-slave-6ntk krb5kdc[1649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157820, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:00.183667 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.169494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.0.205.129:44063)
0808 02:57:00.169685 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:00.169690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:00.170211 (+ 521us) server_negotiation.cc:407] Connection header received
0808 02:57:00.170377 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:00.170381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:00.170459 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:00.170555 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:00.171870 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.172625 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.173861 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.174327 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.174428 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.178467 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:00.178473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:00.178476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:00.178512 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:00.181199 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.181770 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.181773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.181776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.182004 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.182422 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.182427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.182429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.182749 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:00.182961 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:00.183194 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 02:57:00.183361 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 02:57:00 dist-test-slave-6ntk krb5kdc[1649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:57:00 dist-test-slave-6ntk krb5kdc[1649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:00.241109 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.226298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51490 (local address 127.0.205.129:44063)
0808 02:57:00.226532 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:00.226541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:00.226916 (+ 375us) server_negotiation.cc:407] Connection header received
0808 02:57:00.227149 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:00.227155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:00.227232 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:00.227347 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:00.228844 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.229575 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.230985 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.231442 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.231584 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.236045 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:00.236050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:00.236054 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:00.236089 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:00.238793 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.239510 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.239514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.239516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.239643 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.239954 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.239957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.239959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.240211 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:00.240420 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:00.240703 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 02:57:00.240870 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 02:57:00.243496 1874 tablet_service.cc:2908] Rejecting scan request for tablet 83fc9276fa0b4619893a71a74fbc4661: Uninitialized: clean time has not yet been initialized
I20260808 02:57:00.247262 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1817
I20260808 02:57:00.260511 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.248252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51504 (local address 127.0.205.129:44063)
0808 02:57:00.248412 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:00.248417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:00.248833 (+ 416us) server_negotiation.cc:407] Connection header received
0808 02:57:00.248984 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:00.248988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:00.249046 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:00.249127 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:00.250394 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.251100 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.252346 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.252813 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.252931 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.255349 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:00.255354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:00.255357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:00.255391 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:00.258319 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.258931 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.258935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.258937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.259052 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.259369 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.259373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.259374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.259634 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:00.259829 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:00.260096 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 02:57:00.260264 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:57:00.261198 2001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:00.355253 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1817
I20260808 02:57:00.379860 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2018
Aug 08 02:57:00 dist-test-slave-6ntk krb5kdc[1649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:57:00.396057 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.380740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35088 (local address 127.0.205.130:39265)
0808 02:57:00.380933 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:00.380940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:00.381014 (+ 74us) server_negotiation.cc:407] Connection header received
0808 02:57:00.381257 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:00.381262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:00.381329 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:00.381450 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:00.382763 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.383768 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.385147 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.385577 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.385718 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.389804 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:00.389820 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:00.389836 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:00.389914 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:00.392753 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.393551 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.393559 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.393562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.393701 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.394317 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.394323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.394328 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.394766 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:00.395133 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:00.395451 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 02:57:00.395702 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:57:00.396828 2189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:00.490314 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 2018
W20260808 02:57:00.514889 2232 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.205.130:39265 (error 108)
I20260808 02:57:00.515375 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2209
I20260808 02:57:00.517309 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.516520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:39265 (local address 127.0.205.131:37119)
0808 02:57:00.516757 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:00.517015 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.130:39265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260808 02:57:00.517705 2232 consensus_peers.cc:487] T 83fc9276fa0b4619893a71a74fbc4661 P be32b1a8ef9b45bab83c0fa097aa08e1 -> Peer 6da8bbc41e9f47d3b551082000a95f37 (127.0.205.130:39265): Couldn't send request to peer 6da8bbc41e9f47d3b551082000a95f37. Status: Network error: Client connection negotiation failed: client connection to 127.0.205.130:39265: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 08 02:57:00 dist-test-slave-6ntk krb5kdc[1649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:57:00.531876 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:00.516187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47184 (local address 127.0.205.131:35143)
0808 02:57:00.516340 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:00.516348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:00.516373 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:57:00.516572 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:00.516578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:00.516658 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:00.516757 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:00.518479 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.519223 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.520480 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:00.520871 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:00.520969 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:00.526018 (+ 5049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:00.526034 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:00.526051 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:00.526089 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:00.529234 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.530022 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.530030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.530032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.530164 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:00.530551 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:00.530555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:00.530557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:00.530936 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:00.531197 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:00.531440 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 02:57:00.531615 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:57:00.532542 2399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:00.625710 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 2209
I20260808 02:57:00.649281 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1670
I20260808 02:57:00.650704 1766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:00.763721 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1670
2026-08-08T02:57:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4487 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:57:00 dist-test-slave-6ntk krb5kdc[2472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:57:00 dist-test-slave-6ntk krb5kdc[2472](info): set up 2 sockets
Aug 08 02:57:00 dist-test-slave-6ntk krb5kdc[2472](info): commencing operation
krb5kdc: starting...
W20260808 02:57:02.116529 822 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.004s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:57:02 dist-test-slave-6ntk krb5kdc[2472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:57:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:57:02.334172 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:40643
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:40643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:02.673918 2493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:02.674314 2493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:02.674484 2493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:02.681694 2493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:02.681857 2493 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:57:02.681975 2493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:02.682098 2493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:02.682196 2493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:02.682294 2493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:02.692243 2493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:40643
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:40643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:02.693641 2493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:02.695055 2493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:02.707448 2500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:02.707403 2502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:02.707366 2505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:02.708495 2493 server_base.cc:1034] running on GCE node
I20260808 02:57:02.709362 2493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:02.710852 2493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:02.712081 2493 hybrid_clock.cc:648] HybridClock initialized: now 1786157822712008 us; error 48 us; skew 500 ppm
I20260808 02:57:02.712453 2493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:02 dist-test-slave-6ntk krb5kdc[2472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157822, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:02.715901 2493 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:57:02.717404 2493 webserver.cc:460] Webserver started at http://127.0.205.190:45097/ using document root <none> and password file <none>
I20260808 02:57:02.718207 2493 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:02.718326 2493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:02.718690 2493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:02.721719 2493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "59d8a29468eb45f5ad3238554ab2e257"
format_stamp: "Formatted at 2026-08-08 02:57:02 on dist-test-slave-6ntk"
server_key: "301d58d1fd2c898ddf13129c51eccd1d"
server_key_iv: "da1388c191d0e4c7a34dd9e5218ea82f"
server_key_version: "encryptionkey@0"
I20260808 02:57:02.722615 2493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "59d8a29468eb45f5ad3238554ab2e257"
format_stamp: "Formatted at 2026-08-08 02:57:02 on dist-test-slave-6ntk"
server_key: "301d58d1fd2c898ddf13129c51eccd1d"
server_key_iv: "da1388c191d0e4c7a34dd9e5218ea82f"
server_key_version: "encryptionkey@0"
I20260808 02:57:02.727882 2493 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:57:02.731369 2515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:02.732702 2493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:57:02.732895 2493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "59d8a29468eb45f5ad3238554ab2e257"
format_stamp: "Formatted at 2026-08-08 02:57:02 on dist-test-slave-6ntk"
server_key: "301d58d1fd2c898ddf13129c51eccd1d"
server_key_iv: "da1388c191d0e4c7a34dd9e5218ea82f"
server_key_version: "encryptionkey@0"
I20260808 02:57:02.733109 2493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:02.744436 2493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:02.748502 2493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:02.748723 2493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:02.771152 2493 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:40643
I20260808 02:57:02.771168 2619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:40643 every 8 connection(s)
I20260808 02:57:02.772610 2493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:02.774446 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 2493
I20260808 02:57:02.774674 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:02.775372 822 external_mini_cluster.cc:1428] Setting key 1a3772fbd706a3a7f53938b67bc6e737
I20260808 02:57:02.777236 2621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:57:02 dist-test-slave-6ntk krb5kdc[2472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:02.791498 2621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257: Bootstrap starting.
I20260808 02:57:02.795919 2621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:02.797255 2621 log.cc:824] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:02.800977 2621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257: No bootstrap required, opened a new log
I20260808 02:57:02.801196 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:02.777495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.0.205.190:40643)
0808 02:57:02.778378 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:02.778404 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:02.778458 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:57:02.780571 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:02.780602 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:02.781142 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:02.781621 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:02.783051 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:02.785182 (+ 2131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:02.786595 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:02.787641 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:02.787840 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:02.792292 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:02.792303 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:02.792322 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:02.792369 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:02.796533 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:02.797252 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:02.797262 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:02.797272 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:02.797494 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:02.797941 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:02.797949 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:02.797960 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:02.798577 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:02.798854 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:02.799311 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 02:57:02.799778 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":398,"threads_started":1}
I20260808 02:57:02.809419 2621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d8a29468eb45f5ad3238554ab2e257" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40643 } }
I20260808 02:57:02.809777 2621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:02.809923 2621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59d8a29468eb45f5ad3238554ab2e257, State: Initialized, Role: FOLLOWER
I20260808 02:57:02.810637 2621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d8a29468eb45f5ad3238554ab2e257" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40643 } }
I20260808 02:57:02.810935 2621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:02.811084 2621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:02.811272 2621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:02.813531 2621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d8a29468eb45f5ad3238554ab2e257" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40643 } }
I20260808 02:57:02.814118 2621 leader_election.cc:302] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59d8a29468eb45f5ad3238554ab2e257; no voters:
I20260808 02:57:02.814711 2621 leader_election.cc:288] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:02.814898 2626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:02.815662 2626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [term 1 LEADER]: Becoming Leader. State: Replica: 59d8a29468eb45f5ad3238554ab2e257, State: Running, Role: LEADER
I20260808 02:57:02.816232 2626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d8a29468eb45f5ad3238554ab2e257" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40643 } }
I20260808 02:57:02.816946 2621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:02.819867 2628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59d8a29468eb45f5ad3238554ab2e257. Latest consensus state: current_term: 1 leader_uuid: "59d8a29468eb45f5ad3238554ab2e257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d8a29468eb45f5ad3238554ab2e257" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40643 } } }
I20260808 02:57:02.820359 2628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:02.820879 2627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59d8a29468eb45f5ad3238554ab2e257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59d8a29468eb45f5ad3238554ab2e257" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40643 } } }
I20260808 02:57:02.821225 2627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:02.823439 2638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:02.828126 2638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:02.837121 2638 catalog_manager.cc:1347] Generated new cluster ID: bf5258dd13474c7da0c8e05d00b81680
I20260808 02:57:02.837250 2638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:02.864058 2638 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:02.866222 2638 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:02.873332 2638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257: Generated new TSK 0
I20260808 02:57:02.874303 2638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:57:02.934994 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:40643
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:03.313102 2659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:03.313421 2659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:03.313602 2659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:03.321161 2659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:03.321282 2659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:03.321369 2659 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:57:03.321445 2659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:03.331979 2659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.205.190:40643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:03.333279 2659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:03.334704 2659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:03.348765 2664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:03.348769 2665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:03.348793 2669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:03.349815 2659 server_base.cc:1034] running on GCE node
I20260808 02:57:03.350584 2659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:03.351639 2659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:03.352905 2659 hybrid_clock.cc:648] HybridClock initialized: now 1786157823352821 us; error 75 us; skew 500 ppm
I20260808 02:57:03.353228 2659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:03 dist-test-slave-6ntk krb5kdc[2472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:03.356689 2659 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:57:03.358354 2659 webserver.cc:460] Webserver started at http://127.0.205.129:40675/ using document root <none> and password file <none>
I20260808 02:57:03.359126 2659 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:03.359266 2659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:03.359622 2659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:03.362787 2659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "e7ed3c779c02428ba07bd9e83cebac05"
format_stamp: "Formatted at 2026-08-08 02:57:03 on dist-test-slave-6ntk"
server_key: "0a36c11189086e1a402c1fea8292f0df"
server_key_iv: "2aa1f3cce06765c050a3ccdbeeed5a79"
server_key_version: "encryptionkey@0"
I20260808 02:57:03.363648 2659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "e7ed3c779c02428ba07bd9e83cebac05"
format_stamp: "Formatted at 2026-08-08 02:57:03 on dist-test-slave-6ntk"
server_key: "0a36c11189086e1a402c1fea8292f0df"
server_key_iv: "2aa1f3cce06765c050a3ccdbeeed5a79"
server_key_version: "encryptionkey@0"
I20260808 02:57:03.369289 2659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:57:03.373087 2684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:03.374604 2659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:03.374794 2659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "e7ed3c779c02428ba07bd9e83cebac05"
format_stamp: "Formatted at 2026-08-08 02:57:03 on dist-test-slave-6ntk"
server_key: "0a36c11189086e1a402c1fea8292f0df"
server_key_iv: "2aa1f3cce06765c050a3ccdbeeed5a79"
server_key_version: "encryptionkey@0"
I20260808 02:57:03.375038 2659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:03.391117 2659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:03.395929 2659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:03.396229 2659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:03.399492 2659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:03.399619 2659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:03.399835 2659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:03.399921 2659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:57:03 dist-test-slave-6ntk krb5kdc[2472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:03.426497 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:03.400697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:40643 (local address 127.0.205.129:52653)
0808 02:57:03.401574 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:03.401621 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:03.403614 (+ 1993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:03.404492 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:03.404504 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:03.405341 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:03.406452 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:03.406487 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.408007 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.408022 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:03.410325 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:03.410339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.411317 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.411326 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:03.411642 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:03.412591 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:03.412633 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:03.416202 (+ 3569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:03.420538 (+ 4336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:03.420547 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:03.420577 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:03.421315 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:03.421836 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:03.421841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:03.421845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:03.422283 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:03.423013 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:03.423021 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:03.423599 (+ 578us) client_negotiation.cc:769] Sending connection context
0808 02:57:03.423979 (+ 380us) client_negotiation.cc:240] Negotiation successful
0808 02:57:03.424609 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":382,"threads_started":1}
I20260808 02:57:03.426692 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:03.400009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:52653 (local address 127.0.205.190:40643)
0808 02:57:03.400957 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:03.400962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:03.401754 (+ 792us) server_negotiation.cc:407] Connection header received
0808 02:57:03.403961 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:03.403967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:03.404065 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:03.404209 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:03.406681 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.407821 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.410581 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.411050 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.411203 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:03.416534 (+ 5331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:03.416539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:03.416542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:03.416577 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:03.420367 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:03.421522 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:03.421528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:03.421532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:03.421699 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:03.422432 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:03.422438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:03.422441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:03.422875 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:03.423030 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:03.426150 (+ 3120us) server_negotiation.cc:299] Negotiation successful
0808 02:57:03.426412 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":715,"threads_started":1}
I20260808 02:57:03.445571 2659 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:42927
I20260808 02:57:03.445585 2852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:42927 every 8 connection(s)
I20260808 02:57:03.447517 2659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:03.449777 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 2659
I20260808 02:57:03.450042 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:57:03.450842 822 external_mini_cluster.cc:1428] Setting key 201ceb3ba32244306a0635c0a8b8daf5
I20260808 02:57:03.455888 2856 heartbeater.cc:347] Connected to a master server at 127.0.205.190:40643
I20260808 02:57:03.456357 2856 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:03.457600 2856 heartbeater.cc:511] Master 127.0.205.190:40643 requested a full tablet report, sending...
I20260808 02:57:03.461452 2550 ts_manager.cc:194] Registered new tserver with Master: e7ed3c779c02428ba07bd9e83cebac05 (127.0.205.129:42927)
I20260808 02:57:03.463133 2550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:52653
I20260808 02:57:03.465245 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:03.446678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:47719 (local address 127.0.205.190:40643)
0808 02:57:03.446920 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:03.446928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:03.447538 (+ 610us) server_negotiation.cc:407] Connection header received
0808 02:57:03.447805 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:03.447809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:03.447880 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:03.447991 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:03.449622 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.450832 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.453454 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.454241 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.454404 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:03.457605 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:03.457615 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:03.457622 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:03.457671 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:03.461750 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:03.462749 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:03.462758 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:03.462765 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:03.462934 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:03.463411 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:03.463418 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:03.463425 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:03.463840 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:03.463993 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:03.464652 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 02:57:03.464921 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:57:03.465553 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:03.446674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:40643 (local address 127.0.205.129:47719)
0808 02:57:03.447404 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:03.447427 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:03.447603 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:03.448074 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:03.448083 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:03.448899 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:03.449416 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:03.449434 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.451082 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.451090 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:03.453258 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:03.453279 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.454700 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.454710 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:03.454972 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:03.455954 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:03.455983 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:03.457356 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:03.461977 (+ 4621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:03.461985 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:03.461991 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:03.462604 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:03.463092 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:03.463099 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:03.463106 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:03.463275 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:03.464099 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:03.464106 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:03.464337 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 02:57:03.464519 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 02:57:03.464932 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":397,"threads_started":1}
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:57:03.505946 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:40643
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:03.862883 2860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:03.863296 2860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:03.863487 2860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:03.871349 2860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:03.871533 2860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:03.871694 2860 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:57:03.871850 2860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:03.882561 2860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.205.190:40643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:03.884024 2860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:03.885489 2860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:03.900306 2866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:03.900249 2868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:03.900278 2871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:03.901387 2860 server_base.cc:1034] running on GCE node
I20260808 02:57:03.902174 2860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:03.903239 2860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:03.904523 2860 hybrid_clock.cc:648] HybridClock initialized: now 1786157823904437 us; error 84 us; skew 500 ppm
I20260808 02:57:03.904874 2860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:03 dist-test-slave-6ntk krb5kdc[2472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:03.908598 2860 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:57:03.910617 2860 webserver.cc:460] Webserver started at http://127.0.205.130:33827/ using document root <none> and password file <none>
I20260808 02:57:03.911466 2860 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:03.911619 2860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:03.911993 2860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:03.915423 2860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "34a69043582a424cabe0e3b6daeac2c6"
format_stamp: "Formatted at 2026-08-08 02:57:03 on dist-test-slave-6ntk"
server_key: "fb98a0d0d735cac1d5fc2c68116217fa"
server_key_iv: "4bfb7aeb40ad47b1b32b40b34897fb47"
server_key_version: "encryptionkey@0"
I20260808 02:57:03.916363 2860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "34a69043582a424cabe0e3b6daeac2c6"
format_stamp: "Formatted at 2026-08-08 02:57:03 on dist-test-slave-6ntk"
server_key: "fb98a0d0d735cac1d5fc2c68116217fa"
server_key_iv: "4bfb7aeb40ad47b1b32b40b34897fb47"
server_key_version: "encryptionkey@0"
I20260808 02:57:03.924892 2860 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260808 02:57:03.930516 2879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:03.932423 2860 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:57:03.932651 2860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "34a69043582a424cabe0e3b6daeac2c6"
format_stamp: "Formatted at 2026-08-08 02:57:03 on dist-test-slave-6ntk"
server_key: "fb98a0d0d735cac1d5fc2c68116217fa"
server_key_iv: "4bfb7aeb40ad47b1b32b40b34897fb47"
server_key_version: "encryptionkey@0"
I20260808 02:57:03.932993 2860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:03.970564 2860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:03.975822 2860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:03.976168 2860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:03.979413 2860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:03.979512 2860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:03.979713 2860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:03.979809 2860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:57:03 dist-test-slave-6ntk krb5kdc[2472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:04.005048 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:03.980116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:59539 (local address 127.0.205.190:40643)
0808 02:57:03.980814 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:03.980820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:03.981970 (+ 1150us) server_negotiation.cc:407] Connection header received
0808 02:57:03.984067 (+ 2097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:03.984076 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:03.984183 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:03.984354 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:03.986812 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.988272 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.991123 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.991558 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.991703 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:03.996735 (+ 5032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:03.996740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:03.996744 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:03.996779 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:04.000361 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:04.001500 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:04.001506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:04.001509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:04.001648 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:04.002394 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:04.002404 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:04.002411 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:04.002984 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:04.003255 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.004017 (+ 762us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.004378 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":439,"threads_started":1}
I20260808 02:57:04.005511 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:03.980847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:40643 (local address 127.0.205.130:59539)
0808 02:57:03.981751 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:03.981804 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:03.983737 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:03.984673 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:03.984685 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:03.985568 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:03.986498 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:03.986523 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.988521 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.988529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:03.990963 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:03.990976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:03.991825 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:03.992200 (+ 375us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:03.992428 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:03.993295 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:03.993341 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:03.996485 (+ 3144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:04.000561 (+ 4076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:04.000569 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:04.000584 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:04.001332 (+ 748us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:04.001770 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:04.001776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:04.001779 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:04.002183 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:04.003159 (+ 976us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:04.003166 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:04.003559 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 02:57:04.003795 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 02:57:04.004230 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":389,"threads_started":1}
I20260808 02:57:04.028942 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.012718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:34483 (local address 127.0.205.190:40643)
0808 02:57:04.012908 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:04.012913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:04.013648 (+ 735us) server_negotiation.cc:407] Connection header received
0808 02:57:04.013919 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:04.013926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:04.014008 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:04.014114 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:04.015682 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.016767 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.018523 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.019092 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.019233 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.021762 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:04.021767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:04.021770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:04.021803 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:04.025173 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:04.026461 (+ 1288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:04.026470 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:04.026476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:04.026651 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:04.027116 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:04.027120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:04.027122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:04.027492 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:04.027614 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.028379 (+ 765us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.028602 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:57:04.029548 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.012808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:40643 (local address 127.0.205.130:34483)
0808 02:57:04.013498 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:04.013566 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:04.013722 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:04.014140 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:04.014144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:04.014934 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:04.015440 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.015460 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.016953 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.016959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.018352 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.018365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.019313 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.019321 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.019521 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.020274 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:04.020294 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:04.021508 (+ 1214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:04.025360 (+ 3852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:04.025371 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:04.025378 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:04.026299 (+ 921us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:04.026795 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:04.026800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:04.026804 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:04.026988 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:04.027627 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:04.027634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:04.027937 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 02:57:04.028249 (+ 312us) client_negotiation.cc:240] Negotiation successful
0808 02:57:04.028796 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":413,"threads_started":1}
I20260808 02:57:04.043512 2860 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:36787
I20260808 02:57:04.043535 3059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:36787 every 8 connection(s)
I20260808 02:57:04.044958 2860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:04.047437 3062 heartbeater.cc:347] Connected to a master server at 127.0.205.190:40643
I20260808 02:57:04.047708 3062 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:04.048625 3062 heartbeater.cc:511] Master 127.0.205.190:40643 requested a full tablet report, sending...
I20260808 02:57:04.050238 2550 ts_manager.cc:194] Registered new tserver with Master: 34a69043582a424cabe0e3b6daeac2c6 (127.0.205.130:36787)
I20260808 02:57:04.050470 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 2860
I20260808 02:57:04.050679 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:57:04.051224 822 external_mini_cluster.cc:1428] Setting key d1b28afafd1fe0ebffd606423b483dd0
I20260808 02:57:04.051463 2550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:59539
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:57:04.098392 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:40643
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:04.454514 3067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:04.454810 3067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:04.454979 3067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:04.462713 3067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:04.462857 3067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:04.463016 3067 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:57:04.463111 3067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:04.466955 2856 heartbeater.cc:503] Master 127.0.205.190:40643 was elected leader, sending a full tablet report...
I20260808 02:57:04.473955 3067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.205.190:40643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:04.475360 3067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:04.476768 3067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:04.491066 3073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:04.491146 3074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:04.491104 3078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:04.492081 3067 server_base.cc:1034] running on GCE node
I20260808 02:57:04.492758 3067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:04.493749 3067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:04.494978 3067 hybrid_clock.cc:648] HybridClock initialized: now 1786157824494889 us; error 60 us; skew 500 ppm
I20260808 02:57:04.495309 3067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:04 dist-test-slave-6ntk krb5kdc[2472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:04.498768 3067 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:57:04.500466 3067 webserver.cc:460] Webserver started at http://127.0.205.131:38351/ using document root <none> and password file <none>
I20260808 02:57:04.501279 3067 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:04.501416 3067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:04.501783 3067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:04.504999 3067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "c0fa833ff0dd41acac92136fc4cb8721"
format_stamp: "Formatted at 2026-08-08 02:57:04 on dist-test-slave-6ntk"
server_key: "466359583550f6f4ac8e7808d6928dcc"
server_key_iv: "e2d419564d4ade988bff218f7bd0cbfd"
server_key_version: "encryptionkey@0"
I20260808 02:57:04.505971 3067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "c0fa833ff0dd41acac92136fc4cb8721"
format_stamp: "Formatted at 2026-08-08 02:57:04 on dist-test-slave-6ntk"
server_key: "466359583550f6f4ac8e7808d6928dcc"
server_key_iv: "e2d419564d4ade988bff218f7bd0cbfd"
server_key_version: "encryptionkey@0"
I20260808 02:57:04.512169 3067 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260808 02:57:04.516623 3088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:04.518225 3067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:04.518407 3067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "c0fa833ff0dd41acac92136fc4cb8721"
format_stamp: "Formatted at 2026-08-08 02:57:04 on dist-test-slave-6ntk"
server_key: "466359583550f6f4ac8e7808d6928dcc"
server_key_iv: "e2d419564d4ade988bff218f7bd0cbfd"
server_key_version: "encryptionkey@0"
I20260808 02:57:04.518653 3067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:04.546940 3067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:04.551735 3067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:04.551992 3067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:04.554920 3067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:04.555049 3067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:04.555212 3067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:04.555296 3067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:57:04 dist-test-slave-6ntk krb5kdc[2472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:04.584909 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.555326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:42277 (local address 127.0.205.190:40643)
0808 02:57:04.558545 (+ 3219us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:04.558549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:04.558570 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:57:04.561146 (+ 2576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:04.561153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:04.561259 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:04.561428 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:04.564005 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.565183 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.567735 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.568594 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.568771 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.575165 (+ 6394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:04.575173 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:04.575178 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:04.575235 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:04.579911 (+ 4676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:04.581498 (+ 1587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:04.581507 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:04.581514 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:04.581725 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:04.582488 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:04.582496 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:04.582500 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:04.582949 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:04.583127 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.584219 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.584492 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3113,"thread_start_us":2973,"threads_started":1}
I20260808 02:57:04.586582 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.556175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:40643 (local address 127.0.205.131:42277)
0808 02:57:04.557454 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:04.557521 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:04.560688 (+ 3167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:04.561745 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:04.561761 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:04.562742 (+ 981us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:04.563742 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.563778 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.565402 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.565409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.567530 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.567552 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.568920 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.568931 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.569329 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.570357 (+ 1028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:04.570400 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:04.574697 (+ 4297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:04.580239 (+ 5542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:04.580255 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:04.580290 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:04.581267 (+ 977us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:04.581952 (+ 685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:04.581960 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:04.581966 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:04.582298 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:04.583104 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:04.583114 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:04.583712 (+ 598us) client_negotiation.cc:769] Sending connection context
0808 02:57:04.584118 (+ 406us) client_negotiation.cc:240] Negotiation successful
0808 02:57:04.584790 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"thread_start_us":597,"threads_started":1}
I20260808 02:57:04.597905 3067 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:43463
I20260808 02:57:04.597971 3262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:43463 every 8 connection(s)
I20260808 02:57:04.599254 3067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:57:04.601648 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 3067
I20260808 02:57:04.601727 3263 heartbeater.cc:347] Connected to a master server at 127.0.205.190:40643
I20260808 02:57:04.601836 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:57:04.602015 3263 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:04.602303 822 external_mini_cluster.cc:1428] Setting key 6c4973721f7adcde86a45222fcb8a7e6
I20260808 02:57:04.602936 3263 heartbeater.cc:511] Master 127.0.205.190:40643 requested a full tablet report, sending...
I20260808 02:57:04.605023 2550 ts_manager.cc:194] Registered new tserver with Master: c0fa833ff0dd41acac92136fc4cb8721 (127.0.205.131:43463)
I20260808 02:57:04.606153 2550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:42277
I20260808 02:57:04.610985 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.595609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:45363 (local address 127.0.205.190:40643)
0808 02:57:04.595832 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:04.595840 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:04.596549 (+ 709us) server_negotiation.cc:407] Connection header received
0808 02:57:04.596789 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:04.596795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:04.596884 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:04.597014 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:04.598362 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.599102 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.600827 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.601461 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.601589 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.604552 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:04.604563 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:04.604566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:04.604603 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:04.608032 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:04.608926 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:04.608930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:04.608932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:04.609074 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:04.609494 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:04.609499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:04.609502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:04.609766 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:04.609988 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.610369 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.610618 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 02:57:04.611084 3255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.595678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:40643 (local address 127.0.205.131:45363)
0808 02:57:04.596375 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:04.596408 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:04.596605 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:04.597048 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:04.597053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:04.597701 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:04.598188 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.598203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.599264 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.599270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.600687 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.600703 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.602043 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.602050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.602298 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.603216 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:04.603254 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:04.604371 (+ 1117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:04.608203 (+ 3832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:04.608208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:04.608212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:04.608808 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:04.609230 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:04.609235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:04.609238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:04.609375 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:04.609933 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:04.609937 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:04.610122 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 02:57:04.610274 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 02:57:04.610612 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":359,"threads_started":1}
I20260808 02:57:04.618345 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:57:04 dist-test-slave-6ntk krb5kdc[2472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157824, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:57:04 dist-test-slave-6ntk krb5kdc[2472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157824, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:04.672047 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.657206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60218 (local address 127.0.205.190:40643)
0808 02:57:04.657463 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:04.657472 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:04.657867 (+ 395us) server_negotiation.cc:407] Connection header received
0808 02:57:04.658152 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:04.658157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:04.658225 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:04.658342 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:04.659832 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.660570 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.661931 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.662328 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.662485 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.666586 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:04.666591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:04.666594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:04.666632 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:04.669109 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:04.669781 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:04.669786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:04.669790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:04.669997 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:04.670407 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:04.670416 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:04.670419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:04.670722 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:04.671016 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.671408 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.671652 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 02:57:04.676434 2550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:57:04.683864 2550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:04.710357 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.697188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.0.205.129:42927)
0808 02:57:04.700425 (+ 3237us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:04.700435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:04.701074 (+ 639us) server_negotiation.cc:407] Connection header received
0808 02:57:04.701185 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:04.701194 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:04.701442 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:04.701632 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:04.702881 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.704406 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.707103 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.708661 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.708823 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.709089 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.709778 (+ 689us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.710023 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3079,"thread_start_us":2910,"threads_started":1}
I20260808 02:57:04.710625 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.697151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:36787 (local address 127.0.0.1:48424)
0808 02:57:04.699028 (+ 1877us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:04.699081 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:04.699321 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:04.700806 (+ 1485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:04.700817 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:04.700855 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:04.702096 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.702112 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.704040 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.704049 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.707423 (+ 3374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.707440 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.709310 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.709315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.709824 (+ 509us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.709854 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 02:57:04.709964 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 02:57:04.710181 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1495,"mutex_wait_us":366,"thread_start_us":1350,"threads_started":1}
I20260808 02:57:04.711267 3284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.697387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:42927 (local address 127.0.0.1:58344)
0808 02:57:04.700780 (+ 3393us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:04.700800 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:04.700933 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:04.701889 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:04.701894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:04.701921 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:04.702660 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.702674 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.704573 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.704579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.706928 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.706947 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.709229 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.709236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.709471 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.709529 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 02:57:04.710694 (+ 1165us) client_negotiation.cc:240] Negotiation successful
0808 02:57:04.710844 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3265,"mutex_wait_us":23,"thread_start_us":3115,"threads_started":1}
I20260808 02:57:04.711614 3285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.700941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43463 (local address 127.0.0.1:33214)
0808 02:57:04.701669 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:04.701689 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:04.701803 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:04.702947 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:04.702952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:04.702979 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:04.703349 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.703361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.705978 (+ 2617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.706006 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.708231 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.708244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.710603 (+ 2359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.710613 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.710836 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.710873 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:57:04.711018 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 02:57:04.711171 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":1,"thread_start_us":933,"threads_started":1}
I20260808 02:57:04.711968 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.697490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48424 (local address 127.0.205.130:36787)
0808 02:57:04.700014 (+ 2524us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:04.700028 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:04.700065 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:57:04.700195 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:04.700203 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:04.700477 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:04.702026 (+ 1549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:04.702259 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.703895 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.707618 (+ 3723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.709088 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.709453 (+ 365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.709672 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.711158 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.711520 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2354,"thread_start_us":2234,"threads_started":1}
I20260808 02:57:04.712072 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.701587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.0.205.131:43463)
0808 02:57:04.702461 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:04.702467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:04.702492 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:57:04.702563 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:04.702568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:04.702724 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:04.702940 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:04.703583 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.705247 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.708471 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.710030 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.710181 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.710403 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.711065 (+ 662us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.711273 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":541,"threads_started":1}
I20260808 02:57:04.715657 2751 tablet_service.cc:1467] Processing CreateTablet for tablet 0b64e9d46b894b039c72d881def62ce5 (DEFAULT_TABLE table=test-table [id=c244b21c18774d7c91d629697d4e23ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:04.715667 2952 tablet_service.cc:1467] Processing CreateTablet for tablet 0b64e9d46b894b039c72d881def62ce5 (DEFAULT_TABLE table=test-table [id=c244b21c18774d7c91d629697d4e23ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:04.715667 3163 tablet_service.cc:1467] Processing CreateTablet for tablet 0b64e9d46b894b039c72d881def62ce5 (DEFAULT_TABLE table=test-table [id=c244b21c18774d7c91d629697d4e23ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:04.717082 2751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b64e9d46b894b039c72d881def62ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:04.717097 2952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b64e9d46b894b039c72d881def62ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:04.717095 3163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b64e9d46b894b039c72d881def62ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:04.730816 3288 tablet_bootstrap.cc:492] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: Bootstrap starting.
I20260808 02:57:04.734490 3288 tablet_bootstrap.cc:654] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:04.734721 3291 tablet_bootstrap.cc:492] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: Bootstrap starting.
I20260808 02:57:04.736752 3288 log.cc:824] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:04.739030 3295 tablet_bootstrap.cc:492] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: Bootstrap starting.
I20260808 02:57:04.742980 3291 tablet_bootstrap.cc:654] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:04.743412 3295 tablet_bootstrap.cc:654] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:04.744129 3291 log.cc:824] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:04.744905 3295 log.cc:824] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:04.745155 3288 tablet_bootstrap.cc:492] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: No bootstrap required, opened a new log
I20260808 02:57:04.745585 3288 ts_tablet_manager.cc:1397] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20260808 02:57:04.747918 3295 tablet_bootstrap.cc:492] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: No bootstrap required, opened a new log
I20260808 02:57:04.748212 3291 tablet_bootstrap.cc:492] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: No bootstrap required, opened a new log
I20260808 02:57:04.748417 3295 ts_tablet_manager.cc:1397] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260808 02:57:04.748499 3291 ts_tablet_manager.cc:1397] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260808 02:57:04.756542 3295 raft_consensus.cc:348] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:04.756958 3295 raft_consensus.cc:374] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:04.757102 3295 raft_consensus.cc:729] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a69043582a424cabe0e3b6daeac2c6, State: Initialized, Role: FOLLOWER
I20260808 02:57:04.757771 3295 consensus_queue.cc:260] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:04.758085 3288 raft_consensus.cc:348] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:04.758562 3288 raft_consensus.cc:374] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:04.758714 3288 raft_consensus.cc:729] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ed3c779c02428ba07bd9e83cebac05, State: Initialized, Role: FOLLOWER
I20260808 02:57:04.759217 3062 heartbeater.cc:503] Master 127.0.205.190:40643 was elected leader, sending a full tablet report...
I20260808 02:57:04.759500 3288 consensus_queue.cc:260] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:04.759907 3295 ts_tablet_manager.cc:1428] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260808 02:57:04.760726 3291 raft_consensus.cc:348] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:04.761210 3291 raft_consensus.cc:374] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:04.761315 3291 raft_consensus.cc:729] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0fa833ff0dd41acac92136fc4cb8721, State: Initialized, Role: FOLLOWER
I20260808 02:57:04.762087 3291 consensus_queue.cc:260] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:04.763114 3288 ts_tablet_manager.cc:1428] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: Time spent starting tablet: real 0.017s user 0.004s sys 0.012s
I20260808 02:57:04.764088 3263 heartbeater.cc:503] Master 127.0.205.190:40643 was elected leader, sending a full tablet report...
I20260808 02:57:04.764966 3291 ts_tablet_manager.cc:1428] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: Time spent starting tablet: real 0.016s user 0.012s sys 0.005s
W20260808 02:57:04.797732 3063 tablet.cc:2369] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:57:04.851220 3264 tablet.cc:2369] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:57:04.893198 3304 raft_consensus.cc:482] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:04.893641 3304 raft_consensus.cc:504] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:04.895584 3304 leader_election.cc:288] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34a69043582a424cabe0e3b6daeac2c6 (127.0.205.130:36787), c0fa833ff0dd41acac92136fc4cb8721 (127.0.205.131:43463)
I20260808 02:57:04.903204 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.896463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:53361 (local address 127.0.205.131:43463)
0808 02:57:04.896683 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:04.896689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:04.897440 (+ 751us) server_negotiation.cc:407] Connection header received
0808 02:57:04.897657 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:04.897662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:04.897720 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:04.897817 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:04.898568 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.899273 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.901321 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.902200 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.902413 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.902494 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.902786 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.902916 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:57:04.903251 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.896471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:42741 (local address 127.0.205.130:36787)
0808 02:57:04.896680 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:04.896686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:04.897532 (+ 846us) server_negotiation.cc:407] Connection header received
0808 02:57:04.897735 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:04.897742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:04.897816 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:04.897941 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:04.898674 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.899453 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.901362 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.902224 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.902414 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.902500 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.902841 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.902981 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:57:04.903365 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.896023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:36787 (local address 127.0.205.129:42741)
0808 02:57:04.897411 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:04.897433 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:04.897603 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:04.897953 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:04.897958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:04.897985 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:04.898510 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.898519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.899603 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.899609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.901176 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.901191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.902527 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.902537 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.902667 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.902690 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:57:04.902798 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 02:57:04.902927 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1210,"thread_start_us":973,"threads_started":1}
I20260808 02:57:04.903621 3306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.895961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43463 (local address 127.0.205.129:53361)
0808 02:57:04.897323 (+ 1362us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:04.897340 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:04.897478 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:04.897834 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:04.897838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:04.897863 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:04.898401 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.898414 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.899403 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.899411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.901147 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:04.901160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.902447 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.902455 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:04.902610 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.902634 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:57:04.902762 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 02:57:04.902914 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1206,"mutex_wait_us":63,"spinlock_wait_cycles":40704,"thread_start_us":847,"threads_started":1}
I20260808 02:57:04.904837 3004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b64e9d46b894b039c72d881def62ce5" candidate_uuid: "e7ed3c779c02428ba07bd9e83cebac05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a69043582a424cabe0e3b6daeac2c6" is_pre_election: true
I20260808 02:57:04.904884 3193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b64e9d46b894b039c72d881def62ce5" candidate_uuid: "e7ed3c779c02428ba07bd9e83cebac05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0fa833ff0dd41acac92136fc4cb8721" is_pre_election: true
I20260808 02:57:04.905507 3004 raft_consensus.cc:2454] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ed3c779c02428ba07bd9e83cebac05 in term 0.
I20260808 02:57:04.905508 3193 raft_consensus.cc:2454] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7ed3c779c02428ba07bd9e83cebac05 in term 0.
I20260808 02:57:04.906319 2687 leader_election.cc:302] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a69043582a424cabe0e3b6daeac2c6, e7ed3c779c02428ba07bd9e83cebac05; no voters:
I20260808 02:57:04.906721 3304 raft_consensus.cc:2781] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:04.906883 3304 raft_consensus.cc:482] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:04.906998 3304 raft_consensus.cc:3037] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:04.909524 3304 raft_consensus.cc:504] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:04.910290 3304 leader_election.cc:288] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [CANDIDATE]: Term 1 election: Requested vote from peers 34a69043582a424cabe0e3b6daeac2c6 (127.0.205.130:36787), c0fa833ff0dd41acac92136fc4cb8721 (127.0.205.131:43463)
I20260808 02:57:04.910949 3004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b64e9d46b894b039c72d881def62ce5" candidate_uuid: "e7ed3c779c02428ba07bd9e83cebac05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a69043582a424cabe0e3b6daeac2c6"
I20260808 02:57:04.911103 3193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b64e9d46b894b039c72d881def62ce5" candidate_uuid: "e7ed3c779c02428ba07bd9e83cebac05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0fa833ff0dd41acac92136fc4cb8721"
I20260808 02:57:04.911166 3004 raft_consensus.cc:3037] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:04.911350 3193 raft_consensus.cc:3037] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:04.913672 3004 raft_consensus.cc:2454] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ed3c779c02428ba07bd9e83cebac05 in term 1.
I20260808 02:57:04.913933 3193 raft_consensus.cc:2454] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7ed3c779c02428ba07bd9e83cebac05 in term 1.
I20260808 02:57:04.914158 2687 leader_election.cc:302] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a69043582a424cabe0e3b6daeac2c6, e7ed3c779c02428ba07bd9e83cebac05; no voters:
I20260808 02:57:04.914501 3304 raft_consensus.cc:2781] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:04.915004 3304 raft_consensus.cc:686] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 1 LEADER]: Becoming Leader. State: Replica: e7ed3c779c02428ba07bd9e83cebac05, State: Running, Role: LEADER
I20260808 02:57:04.915512 3304 consensus_queue.cc:237] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:04.920359 2545 catalog_manager.cc:5495] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7ed3c779c02428ba07bd9e83cebac05 (127.0.205.129). New cstate: current_term: 1 leader_uuid: "e7ed3c779c02428ba07bd9e83cebac05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } health_report { overall_health: UNKNOWN } } }
I20260808 02:57:04.962823 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:04.956664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.0.205.129:42927)
0808 02:57:04.956917 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:04.956926 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:04.956976 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:57:04.957440 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:04.957444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:04.957527 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:04.957681 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:04.958133 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.958842 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.960233 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:04.960702 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:04.960849 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:04.961404 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:04.962118 (+ 714us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:04.962292 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:04.962415 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 02:57:04.962510 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260808 02:57:04.962844 2857 tablet.cc:2369] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:57:04.972493 3193 raft_consensus.cc:1261] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721 [term 1 FOLLOWER]: Refusing update from remote peer e7ed3c779c02428ba07bd9e83cebac05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:04.972442 3004 raft_consensus.cc:1261] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6 [term 1 FOLLOWER]: Refusing update from remote peer e7ed3c779c02428ba07bd9e83cebac05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:04.973997 3304 consensus_queue.cc:1035] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:04.974449 3308 consensus_queue.cc:1035] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:04.989117 3311 mvcc.cc:205] Tried to move back new op lower bound from 7316102451074506752 to 7316102450857668608. Current Snapshot: MvccSnapshot[applied={T|T < 7316102451074506752}]
I20260808 02:57:05.014129 2550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:05.014559 2550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:05.019480 2550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257: Sending DeleteTablet for 3 replicas of tablet 0b64e9d46b894b039c72d881def62ce5
I20260808 02:57:05.021248 2952 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b64e9d46b894b039c72d881def62ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:05 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:48424
I20260808 02:57:05.021284 2751 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b64e9d46b894b039c72d881def62ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:05 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:58344
I20260808 02:57:05.021452 3163 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b64e9d46b894b039c72d881def62ce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:05 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:33214
I20260808 02:57:05.028017 2550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:57:05.029942 2550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:05.034350 3327 tablet_replica.cc:330] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: stopping tablet replica
I20260808 02:57:05.035120 3327 raft_consensus.cc:2229] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:05.035712 3327 raft_consensus.cc:2258] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:05.036921 3325 tablet_replica.cc:330] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: stopping tablet replica
I20260808 02:57:05.039381 3325 raft_consensus.cc:2229] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:05.040046 3325 raft_consensus.cc:2258] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:05.040750 3328 tablet_replica.cc:330] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: stopping tablet replica
I20260808 02:57:05.041245 3328 raft_consensus.cc:2229] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:57:05.042411 3328 raft_consensus.cc:2258] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:05.042351 2952 tablet_service.cc:1467] Processing CreateTablet for tablet c23c8f1076554beb964f0e401c3c495e (DEFAULT_TABLE table=test-table [id=4894e248839b4aafa81b7a63587dddec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:05.043012 2952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c23c8f1076554beb964f0e401c3c495e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:05.043607 3327 ts_tablet_manager.cc:1905] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:05.044446 3325 ts_tablet_manager.cc:1905] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:05.047298 3328 ts_tablet_manager.cc:1905] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:05.052856 3327 ts_tablet_manager.cc:1918] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:05.053298 3327 log.cc:1196] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/0b64e9d46b894b039c72d881def62ce5
I20260808 02:57:05.054190 3327 ts_tablet_manager.cc:1939] T 0b64e9d46b894b039c72d881def62ce5 P 34a69043582a424cabe0e3b6daeac2c6: Deleting consensus metadata
I20260808 02:57:05.055315 3325 ts_tablet_manager.cc:1918] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:05.055488 3325 log.cc:1196] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/0b64e9d46b894b039c72d881def62ce5
I20260808 02:57:05.056011 3325 ts_tablet_manager.cc:1939] T 0b64e9d46b894b039c72d881def62ce5 P c0fa833ff0dd41acac92136fc4cb8721: Deleting consensus metadata
I20260808 02:57:05.057024 2519 catalog_manager.cc:4841] TS 34a69043582a424cabe0e3b6daeac2c6 (127.0.205.130:36787): tablet 0b64e9d46b894b039c72d881def62ce5 (table test-table [id=c244b21c18774d7c91d629697d4e23ac]) successfully deleted
I20260808 02:57:05.058431 3163 tablet_service.cc:1467] Processing CreateTablet for tablet c23c8f1076554beb964f0e401c3c495e (DEFAULT_TABLE table=test-table [id=4894e248839b4aafa81b7a63587dddec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:05.059036 3163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c23c8f1076554beb964f0e401c3c495e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:05.059022 2516 catalog_manager.cc:4841] TS c0fa833ff0dd41acac92136fc4cb8721 (127.0.205.131:43463): tablet 0b64e9d46b894b039c72d881def62ce5 (table test-table [id=c244b21c18774d7c91d629697d4e23ac]) successfully deleted
I20260808 02:57:05.059631 3328 ts_tablet_manager.cc:1918] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:05.059792 3328 log.cc:1196] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/0b64e9d46b894b039c72d881def62ce5
I20260808 02:57:05.060351 3328 ts_tablet_manager.cc:1939] T 0b64e9d46b894b039c72d881def62ce5 P e7ed3c779c02428ba07bd9e83cebac05: Deleting consensus metadata
I20260808 02:57:05.062898 2751 tablet_service.cc:1467] Processing CreateTablet for tablet c23c8f1076554beb964f0e401c3c495e (DEFAULT_TABLE table=test-table [id=4894e248839b4aafa81b7a63587dddec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:05.063545 3295 tablet_bootstrap.cc:492] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6: Bootstrap starting.
I20260808 02:57:05.063609 2751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c23c8f1076554beb964f0e401c3c495e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:05.065369 3295 tablet_bootstrap.cc:654] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:05.066604 3291 tablet_bootstrap.cc:492] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721: Bootstrap starting.
I20260808 02:57:05.067968 2516 catalog_manager.cc:4841] TS e7ed3c779c02428ba07bd9e83cebac05 (127.0.205.129:42927): tablet 0b64e9d46b894b039c72d881def62ce5 (table test-table [id=c244b21c18774d7c91d629697d4e23ac]) successfully deleted
I20260808 02:57:05.068367 3291 tablet_bootstrap.cc:654] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:05.073526 3288 tablet_bootstrap.cc:492] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05: Bootstrap starting.
I20260808 02:57:05.075840 3288 tablet_bootstrap.cc:654] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:05.091917 3291 tablet_bootstrap.cc:492] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721: No bootstrap required, opened a new log
I20260808 02:57:05.091977 3288 tablet_bootstrap.cc:492] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05: No bootstrap required, opened a new log
I20260808 02:57:05.092155 3288 ts_tablet_manager.cc:1397] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.006s
I20260808 02:57:05.092178 3291 ts_tablet_manager.cc:1397] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.008s
I20260808 02:57:05.093065 3291 raft_consensus.cc:348] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.093114 3288 raft_consensus.cc:348] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.093338 3291 raft_consensus.cc:374] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:05.093360 3288 raft_consensus.cc:374] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:05.093457 3291 raft_consensus.cc:729] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0fa833ff0dd41acac92136fc4cb8721, State: Initialized, Role: FOLLOWER
I20260808 02:57:05.093487 3288 raft_consensus.cc:729] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ed3c779c02428ba07bd9e83cebac05, State: Initialized, Role: FOLLOWER
I20260808 02:57:05.093833 3288 consensus_queue.cc:260] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.093829 3291 consensus_queue.cc:260] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.094708 3291 ts_tablet_manager.cc:1428] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:57:05.095537 3288 ts_tablet_manager.cc:1428] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:05.095681 3295 tablet_bootstrap.cc:492] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6: No bootstrap required, opened a new log
I20260808 02:57:05.095858 3295 ts_tablet_manager.cc:1397] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.006s
I20260808 02:57:05.096804 3295 raft_consensus.cc:348] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.097066 3295 raft_consensus.cc:374] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:05.099045 3295 raft_consensus.cc:729] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a69043582a424cabe0e3b6daeac2c6, State: Initialized, Role: FOLLOWER
I20260808 02:57:05.100503 3295 consensus_queue.cc:260] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.101257 3295 ts_tablet_manager.cc:1428] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260808 02:57:05.126919 3301 raft_consensus.cc:482] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:05.127216 3301 raft_consensus.cc:504] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.129146 3301 leader_election.cc:288] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ed3c779c02428ba07bd9e83cebac05 (127.0.205.129:42927), c0fa833ff0dd41acac92136fc4cb8721 (127.0.205.131:43463)
I20260808 02:57:05.137739 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.130358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:45195 (local address 127.0.205.131:43463)
0808 02:57:05.130533 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.130538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.131168 (+ 630us) server_negotiation.cc:407] Connection header received
0808 02:57:05.131344 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.131350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.131431 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.131526 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:05.132164 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.133278 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.135518 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.136490 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.136674 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.136749 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.137197 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.137397 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:57:05.137745 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.130212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43463 (local address 127.0.205.130:45195)
0808 02:57:05.131039 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:05.131057 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:05.131185 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:05.131581 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:05.131585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:05.131616 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:05.131991 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.132003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.133441 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.133447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.135361 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.135373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.136705 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.136711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.137003 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.137045 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 02:57:05.137164 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 02:57:05.137296 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":91,"thread_start_us":458,"threads_started":1}
I20260808 02:57:05.137822 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.129610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:44983 (local address 127.0.205.129:42927)
0808 02:57:05.129846 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.129852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.130183 (+ 331us) server_negotiation.cc:407] Connection header received
0808 02:57:05.130510 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.130515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.130605 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.130684 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:05.131549 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.132584 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.134995 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.136409 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.136577 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.136672 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.137320 (+ 648us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.137471 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:57:05.138723 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.129434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:42927 (local address 127.0.205.130:44983)
0808 02:57:05.130057 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:05.130073 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:05.130311 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:05.130929 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:05.130933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:05.130969 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:05.131370 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.131388 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.132724 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.132731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.134817 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.134833 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.136875 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.136882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.137120 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.137152 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:57:05.138155 (+ 1003us) client_negotiation.cc:240] Negotiation successful
0808 02:57:05.138299 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":19,"thread_start_us":310,"threads_started":1}
I20260808 02:57:05.139446 3193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23c8f1076554beb964f0e401c3c495e" candidate_uuid: "34a69043582a424cabe0e3b6daeac2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0fa833ff0dd41acac92136fc4cb8721" is_pre_election: true
I20260808 02:57:05.139817 3193 raft_consensus.cc:2454] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a69043582a424cabe0e3b6daeac2c6 in term 0.
I20260808 02:57:05.140089 2782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23c8f1076554beb964f0e401c3c495e" candidate_uuid: "34a69043582a424cabe0e3b6daeac2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ed3c779c02428ba07bd9e83cebac05" is_pre_election: true
I20260808 02:57:05.140412 2880 leader_election.cc:302] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a69043582a424cabe0e3b6daeac2c6, c0fa833ff0dd41acac92136fc4cb8721; no voters:
I20260808 02:57:05.140654 2782 raft_consensus.cc:2454] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34a69043582a424cabe0e3b6daeac2c6 in term 0.
I20260808 02:57:05.140923 3301 raft_consensus.cc:2781] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:05.141042 3301 raft_consensus.cc:482] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:05.141131 3301 raft_consensus.cc:3037] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:05.144500 3301 raft_consensus.cc:504] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.145215 3301 leader_election.cc:288] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [CANDIDATE]: Term 1 election: Requested vote from peers e7ed3c779c02428ba07bd9e83cebac05 (127.0.205.129:42927), c0fa833ff0dd41acac92136fc4cb8721 (127.0.205.131:43463)
I20260808 02:57:05.145699 2782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23c8f1076554beb964f0e401c3c495e" candidate_uuid: "34a69043582a424cabe0e3b6daeac2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ed3c779c02428ba07bd9e83cebac05"
I20260808 02:57:05.145975 2782 raft_consensus.cc:3037] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:05.145857 3193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23c8f1076554beb964f0e401c3c495e" candidate_uuid: "34a69043582a424cabe0e3b6daeac2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0fa833ff0dd41acac92136fc4cb8721"
I20260808 02:57:05.146116 3193 raft_consensus.cc:3037] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:05.149075 2782 raft_consensus.cc:2454] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a69043582a424cabe0e3b6daeac2c6 in term 1.
I20260808 02:57:05.149290 3193 raft_consensus.cc:2454] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34a69043582a424cabe0e3b6daeac2c6 in term 1.
I20260808 02:57:05.149585 2880 leader_election.cc:302] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34a69043582a424cabe0e3b6daeac2c6, e7ed3c779c02428ba07bd9e83cebac05; no voters:
I20260808 02:57:05.149925 3301 raft_consensus.cc:2781] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:05.150580 3301 raft_consensus.cc:686] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 1 LEADER]: Becoming Leader. State: Replica: 34a69043582a424cabe0e3b6daeac2c6, State: Running, Role: LEADER
I20260808 02:57:05.151012 3301 consensus_queue.cc:237] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.154690 2545 catalog_manager.cc:5495] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34a69043582a424cabe0e3b6daeac2c6 (127.0.205.130). New cstate: current_term: 1 leader_uuid: "34a69043582a424cabe0e3b6daeac2c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:57:05 dist-test-slave-6ntk krb5kdc[2472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157822, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:05.222620 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.204635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:40643 (local address 127.0.0.1:60222)
0808 02:57:05.205427 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:05.205461 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:05.205759 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:05.206474 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:05.206479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:05.207565 (+ 1086us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:05.207950 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.207967 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.209452 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.209458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.211177 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.211195 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.212250 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.212257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.212448 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.213278 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:05.213305 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:05.216067 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:05.219630 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:05.219646 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:05.219655 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:05.220134 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:05.220530 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:05.220535 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:05.220539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:05.220717 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:05.221281 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:05.221290 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:05.221469 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 02:57:05.221680 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 02:57:05.222057 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":27,"thread_start_us":410,"threads_started":1}
I20260808 02:57:05.222625 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.204635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60222 (local address 127.0.205.190:40643)
0808 02:57:05.206053 (+ 1418us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.206058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.206080 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:57:05.206129 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.206132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.206192 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.206394 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:05.208173 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.209275 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.211358 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.212049 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.212190 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.216281 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:05.216286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:05.216290 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:05.216318 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:05.219484 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:05.220259 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:05.220267 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:05.220270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:05.220418 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:05.220839 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:05.220844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:05.220847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:05.221137 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:05.221298 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.221767 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.222168 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":191,"server-negotiator.queue_time_us":1313,"thread_start_us":1106,"threads_started":1}
I20260808 02:57:05.228345 2545 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:60222:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:57:05.229619 2545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:05.237350 2751 tablet_service.cc:1467] Processing CreateTablet for tablet b66bfbdeaf67457ca2ffbd146fd6be75 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=144f9fa59cbd42f2b3bc495f5d97e633]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:57:05.237484 2952 tablet_service.cc:1467] Processing CreateTablet for tablet b66bfbdeaf67457ca2ffbd146fd6be75 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=144f9fa59cbd42f2b3bc495f5d97e633]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:57:05.238026 2751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b66bfbdeaf67457ca2ffbd146fd6be75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:05.238128 2952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b66bfbdeaf67457ca2ffbd146fd6be75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:05.237955 3163 tablet_service.cc:1467] Processing CreateTablet for tablet b66bfbdeaf67457ca2ffbd146fd6be75 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=144f9fa59cbd42f2b3bc495f5d97e633]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:57:05.238502 3163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b66bfbdeaf67457ca2ffbd146fd6be75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:05.246012 3288 tablet_bootstrap.cc:492] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: Bootstrap starting.
I20260808 02:57:05.246060 3295 tablet_bootstrap.cc:492] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: Bootstrap starting.
I20260808 02:57:05.247653 3295 tablet_bootstrap.cc:654] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:05.247851 3288 tablet_bootstrap.cc:654] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:05.248569 3291 tablet_bootstrap.cc:492] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: Bootstrap starting.
I20260808 02:57:05.250839 3291 tablet_bootstrap.cc:654] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:05.259537 3295 tablet_bootstrap.cc:492] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: No bootstrap required, opened a new log
I20260808 02:57:05.259853 3295 ts_tablet_manager.cc:1397] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260808 02:57:05.261013 3295 raft_consensus.cc:348] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.261473 3295 raft_consensus.cc:374] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:05.261577 3288 tablet_bootstrap.cc:492] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: No bootstrap required, opened a new log
I20260808 02:57:05.261615 3295 raft_consensus.cc:729] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a69043582a424cabe0e3b6daeac2c6, State: Initialized, Role: FOLLOWER
I20260808 02:57:05.261768 3288 ts_tablet_manager.cc:1397] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.007s
I20260808 02:57:05.262099 3295 consensus_queue.cc:260] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.262601 3288 raft_consensus.cc:348] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.262869 3288 raft_consensus.cc:374] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:05.262792 3301 tablet_replica.cc:439] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } } }
I20260808 02:57:05.262975 3288 raft_consensus.cc:729] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ed3c779c02428ba07bd9e83cebac05, State: Initialized, Role: FOLLOWER
I20260808 02:57:05.263078 3301 tablet_replica.cc:442] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:05.263725 3295 ts_tablet_manager.cc:1428] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260808 02:57:05.263247 3288 consensus_queue.cc:260] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.264516 3308 tablet_replica.cc:439] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } } }
I20260808 02:57:05.264869 3308 tablet_replica.cc:442] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:05.265501 3291 tablet_bootstrap.cc:492] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: No bootstrap required, opened a new log
I20260808 02:57:05.265686 3291 ts_tablet_manager.cc:1397] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.008s
I20260808 02:57:05.266541 3291 raft_consensus.cc:348] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.266841 3291 raft_consensus.cc:374] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:05.266932 3291 raft_consensus.cc:729] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0fa833ff0dd41acac92136fc4cb8721, State: Initialized, Role: FOLLOWER
I20260808 02:57:05.267222 3291 consensus_queue.cc:260] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.267771 3288 ts_tablet_manager.cc:1428] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260808 02:57:05.267693 3305 tablet_replica.cc:439] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } } }
I20260808 02:57:05.268067 3305 tablet_replica.cc:442] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:05.269045 3291 ts_tablet_manager.cc:1428] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 02:57:05.527436 3305 raft_consensus.cc:482] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:05.527642 3305 raft_consensus.cc:504] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.529379 3305 leader_election.cc:288] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ed3c779c02428ba07bd9e83cebac05 (127.0.205.129:42927), 34a69043582a424cabe0e3b6daeac2c6 (127.0.205.130:36787)
I20260808 02:57:05.537485 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.529984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:40749 (local address 127.0.205.129:42927)
0808 02:57:05.530259 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.530265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.530286 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:57:05.530485 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.530493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.530585 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.530679 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:05.531367 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.532302 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.535224 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.536430 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.536624 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.536705 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.537132 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.537273 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260808 02:57:05.538024 3351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.529585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:42927 (local address 127.0.205.131:40749)
0808 02:57:05.530128 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:05.530160 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:05.530305 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:05.530779 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:05.530785 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:05.530838 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:05.531193 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.531207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.532487 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.532492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.535037 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.535053 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.536660 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.536665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.536818 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.536846 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 02:57:05.537065 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 02:57:05.537381 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":21,"spinlock_wait_cycles":12544,"thread_start_us":315,"threads_started":1}
I20260808 02:57:05.538338 3353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.530258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:35347 (local address 127.0.205.130:36787)
0808 02:57:05.530781 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.530788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.530808 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:57:05.530958 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.530962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.531016 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.531093 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:05.531830 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.532760 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.535478 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.536799 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.536955 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.537034 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.537825 (+ 791us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.538024 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":327,"threads_started":1}
I20260808 02:57:05.538363 3352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.529794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:36787 (local address 127.0.205.131:35347)
0808 02:57:05.530642 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:05.530687 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:05.530822 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:05.531162 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:05.531168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:05.531197 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:05.531645 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.531666 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.532903 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.532909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.535321 (+ 2412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.535333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.537033 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.537042 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.537441 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.537478 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:57:05.537580 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 02:57:05.537800 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":51,"thread_start_us":629,"threads_started":1}
I20260808 02:57:05.538853 2782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b66bfbdeaf67457ca2ffbd146fd6be75" candidate_uuid: "c0fa833ff0dd41acac92136fc4cb8721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ed3c779c02428ba07bd9e83cebac05" is_pre_election: true
I20260808 02:57:05.539168 2782 raft_consensus.cc:2454] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0fa833ff0dd41acac92136fc4cb8721 in term 0.
I20260808 02:57:05.539129 3004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b66bfbdeaf67457ca2ffbd146fd6be75" candidate_uuid: "c0fa833ff0dd41acac92136fc4cb8721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a69043582a424cabe0e3b6daeac2c6" is_pre_election: true
I20260808 02:57:05.539393 3004 raft_consensus.cc:2454] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0fa833ff0dd41acac92136fc4cb8721 in term 0.
I20260808 02:57:05.539711 3090 leader_election.cc:302] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0fa833ff0dd41acac92136fc4cb8721, e7ed3c779c02428ba07bd9e83cebac05; no voters:
I20260808 02:57:05.540153 3305 raft_consensus.cc:2781] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:05.540273 3305 raft_consensus.cc:482] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:05.540397 3305 raft_consensus.cc:3037] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:05.542702 3305 raft_consensus.cc:504] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.543349 3305 leader_election.cc:288] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [CANDIDATE]: Term 1 election: Requested vote from peers e7ed3c779c02428ba07bd9e83cebac05 (127.0.205.129:42927), 34a69043582a424cabe0e3b6daeac2c6 (127.0.205.130:36787)
I20260808 02:57:05.543907 2782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b66bfbdeaf67457ca2ffbd146fd6be75" candidate_uuid: "c0fa833ff0dd41acac92136fc4cb8721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ed3c779c02428ba07bd9e83cebac05"
I20260808 02:57:05.543912 3004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b66bfbdeaf67457ca2ffbd146fd6be75" candidate_uuid: "c0fa833ff0dd41acac92136fc4cb8721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a69043582a424cabe0e3b6daeac2c6"
I20260808 02:57:05.544111 3004 raft_consensus.cc:3037] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:05.544152 2782 raft_consensus.cc:3037] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:05.546290 3004 raft_consensus.cc:2454] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0fa833ff0dd41acac92136fc4cb8721 in term 1.
I20260808 02:57:05.546403 2782 raft_consensus.cc:2454] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0fa833ff0dd41acac92136fc4cb8721 in term 1.
I20260808 02:57:05.546854 3090 leader_election.cc:302] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0fa833ff0dd41acac92136fc4cb8721, e7ed3c779c02428ba07bd9e83cebac05; no voters:
I20260808 02:57:05.547173 3305 raft_consensus.cc:2781] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:05.547614 3305 raft_consensus.cc:686] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [term 1 LEADER]: Becoming Leader. State: Replica: c0fa833ff0dd41acac92136fc4cb8721, State: Running, Role: LEADER
I20260808 02:57:05.548041 3305 consensus_queue.cc:237] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } }
I20260808 02:57:05.550756 3354 tablet_replica.cc:439] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: TxnStatusTablet state changed. Reason: New leader c0fa833ff0dd41acac92136fc4cb8721. Latest consensus state: current_term: 1 leader_uuid: "c0fa833ff0dd41acac92136fc4cb8721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } } }
I20260808 02:57:05.551276 3354 tablet_replica.cc:442] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:57:05.552206 3359 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:57:05.552543 3359 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:57:05.553090 2545 catalog_manager.cc:5495] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0fa833ff0dd41acac92136fc4cb8721 (127.0.205.131). New cstate: current_term: 1 leader_uuid: "c0fa833ff0dd41acac92136fc4cb8721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } health_report { overall_health: HEALTHY } } }
I20260808 02:57:05.583292 3301 consensus_queue.cc:1035] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:05.590986 3301 consensus_queue.cc:1035] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:05.635834 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.629675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43463 (local address 127.0.0.1:33228)
0808 02:57:05.629942 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:05.629964 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:05.630128 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:05.630588 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:05.630593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:05.630607 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:57:05.630856 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.630866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.632004 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.632010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.633111 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.633127 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.634030 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.634036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.634216 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.634276 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:57:05.635207 (+ 931us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:57:05.635232 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 02:57:05.635323 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 02:57:05.635499 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 02:57:05.635824 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.629758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33228 (local address 127.0.205.131:43463)
0808 02:57:05.629939 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.629946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.630054 (+ 108us) server_negotiation.cc:407] Connection header received
0808 02:57:05.630314 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.630317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.630400 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.630561 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:05.631019 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.631855 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.633267 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.633788 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.634011 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.634528 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:05.635091 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:05.635258 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.635342 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.635435 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 02:57:05.641150 3004 raft_consensus.cc:1261] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6 [term 1 FOLLOWER]: Refusing update from remote peer c0fa833ff0dd41acac92136fc4cb8721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:05.641322 2782 raft_consensus.cc:1261] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05 [term 1 FOLLOWER]: Refusing update from remote peer c0fa833ff0dd41acac92136fc4cb8721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:05.642019 3305 consensus_queue.cc:1035] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:05.642386 3354 consensus_queue.cc:1035] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:05.646188 3338 tablet_replica.cc:439] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: TxnStatusTablet state changed. Reason: New leader c0fa833ff0dd41acac92136fc4cb8721. Latest consensus state: current_term: 1 leader_uuid: "c0fa833ff0dd41acac92136fc4cb8721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } } }
I20260808 02:57:05.646461 3338 tablet_replica.cc:442] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:05.648363 3366 mvcc.cc:205] Tried to move back new op lower bound from 7316102453819240448 to 7316102453449080832. Current Snapshot: MvccSnapshot[applied={T|T < 7316102453819240448}]
I20260808 02:57:05.649978 3374 mvcc.cc:205] Tried to move back new op lower bound from 7316102453819240448 to 7316102453449080832. Current Snapshot: MvccSnapshot[applied={T|T < 7316102453819240448}]
I20260808 02:57:05.657441 3308 tablet_replica.cc:439] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: TxnStatusTablet state changed. Reason: New leader c0fa833ff0dd41acac92136fc4cb8721. Latest consensus state: current_term: 1 leader_uuid: "c0fa833ff0dd41acac92136fc4cb8721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } } }
I20260808 02:57:05.657850 3308 tablet_replica.cc:442] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:05.661325 3338 tablet_replica.cc:439] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0fa833ff0dd41acac92136fc4cb8721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } } }
I20260808 02:57:05.661672 3338 tablet_replica.cc:442] T b66bfbdeaf67457ca2ffbd146fd6be75 P 34a69043582a424cabe0e3b6daeac2c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:05.662464 3305 tablet_replica.cc:439] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0fa833ff0dd41acac92136fc4cb8721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } } }
I20260808 02:57:05.662791 3305 tablet_replica.cc:442] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:57:05.664208 3305 tablet_replica.cc:439] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0fa833ff0dd41acac92136fc4cb8721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } } }
I20260808 02:57:05.664520 3305 tablet_replica.cc:442] T b66bfbdeaf67457ca2ffbd146fd6be75 P c0fa833ff0dd41acac92136fc4cb8721: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:57:05.665390 3308 tablet_replica.cc:439] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0fa833ff0dd41acac92136fc4cb8721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ed3c779c02428ba07bd9e83cebac05" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 42927 } } peers { permanent_uuid: "34a69043582a424cabe0e3b6daeac2c6" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36787 } } peers { permanent_uuid: "c0fa833ff0dd41acac92136fc4cb8721" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43463 } } }
I20260808 02:57:05.665642 3308 tablet_replica.cc:442] T b66bfbdeaf67457ca2ffbd146fd6be75 P e7ed3c779c02428ba07bd9e83cebac05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:05.676931 3353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.670775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.0.205.130:36787)
0808 02:57:05.670987 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.670995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.671459 (+ 464us) server_negotiation.cc:407] Connection header received
0808 02:57:05.671704 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.671710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.671793 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.671961 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:05.672517 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.673340 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.674813 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.675234 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.675375 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.675797 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:05.676343 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:05.676520 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.676577 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.676656 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:57:05.694072 3390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.687848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43463 (local address 127.0.205.130:55423)
0808 02:57:05.688533 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:05.688549 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:05.688666 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:05.689038 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:05.689043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:05.689055 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:57:05.689261 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.689268 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.690842 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.690846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.691922 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.691934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.692980 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.692985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.693106 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.693155 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:57:05.693545 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:57:05.693553 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:57:05.693614 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 02:57:05.693712 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":359,"threads_started":1}
I20260808 02:57:05.694104 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.687985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:55423 (local address 127.0.205.131:43463)
0808 02:57:05.688221 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.688227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.688638 (+ 411us) server_negotiation.cc:407] Connection header received
0808 02:57:05.688795 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.688800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.688888 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.688986 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:05.689419 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.690652 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.692060 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.692830 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.692966 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.693321 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:05.693447 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:05.693536 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.693663 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.693740 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 02:57:05.751410 3353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.742948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:43095 (local address 127.0.205.130:36787)
0808 02:57:05.743195 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.743206 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.744645 (+ 1439us) server_negotiation.cc:407] Connection header received
0808 02:57:05.744836 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.744840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.744924 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.745011 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:05.745697 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.746971 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.749164 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.749691 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.749841 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.750476 (+ 635us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:05.750642 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:05.750750 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.750935 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.751028 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:57:05.751735 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.742915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:36787 (local address 127.0.205.131:43095)
0808 02:57:05.744529 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:05.744551 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:05.744695 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:05.745052 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:05.745057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:05.745071 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:57:05.745494 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.745506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.747208 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.747216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.748975 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:05.748993 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.749959 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.749967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:05.750189 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.750283 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:57:05.750770 (+ 487us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:57:05.750783 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 02:57:05.750890 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 02:57:05.751249 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1408,"thread_start_us":1197,"threads_started":1}
I20260808 02:57:05.805410 2550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:05.805699 2550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:05.809588 2550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59d8a29468eb45f5ad3238554ab2e257: Sending DeleteTablet for 3 replicas of tablet c23c8f1076554beb964f0e401c3c495e
I20260808 02:57:05.810858 2952 tablet_service.cc:1514] Processing DeleteTablet for tablet c23c8f1076554beb964f0e401c3c495e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:05 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:48424
I20260808 02:57:05.810940 2751 tablet_service.cc:1514] Processing DeleteTablet for tablet c23c8f1076554beb964f0e401c3c495e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:05 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:58344
I20260808 02:57:05.813545 3163 tablet_service.cc:1514] Processing DeleteTablet for tablet c23c8f1076554beb964f0e401c3c495e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:05 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:33214
I20260808 02:57:05.817325 3403 tablet_replica.cc:330] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6: stopping tablet replica
I20260808 02:57:05.817548 3403 raft_consensus.cc:2229] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:57:05.818236 3403 raft_consensus.cc:2258] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:05.819413 3403 ts_tablet_manager.cc:1905] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:05.827059 3403 ts_tablet_manager.cc:1918] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:57:05.827220 3403 log.cc:1196] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/c23c8f1076554beb964f0e401c3c495e
I20260808 02:57:05.827741 3403 ts_tablet_manager.cc:1939] T c23c8f1076554beb964f0e401c3c495e P 34a69043582a424cabe0e3b6daeac2c6: Deleting consensus metadata
I20260808 02:57:05.828995 2519 catalog_manager.cc:4841] TS 34a69043582a424cabe0e3b6daeac2c6 (127.0.205.130:36787): tablet c23c8f1076554beb964f0e401c3c495e (table test-table [id=4894e248839b4aafa81b7a63587dddec]) successfully deleted
I20260808 02:57:05.829113 3402 tablet_replica.cc:330] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05: stopping tablet replica
I20260808 02:57:05.829334 3402 raft_consensus.cc:2229] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:05.829581 3402 raft_consensus.cc:2258] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:05.830706 3402 ts_tablet_manager.cc:1905] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:05.837951 3402 ts_tablet_manager.cc:1918] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:57:05.838099 3402 log.cc:1196] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/c23c8f1076554beb964f0e401c3c495e
I20260808 02:57:05.838618 3402 ts_tablet_manager.cc:1939] T c23c8f1076554beb964f0e401c3c495e P e7ed3c779c02428ba07bd9e83cebac05: Deleting consensus metadata
I20260808 02:57:05.839073 3405 tablet_replica.cc:330] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721: stopping tablet replica
I20260808 02:57:05.839356 3405 raft_consensus.cc:2229] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:05.839860 3405 raft_consensus.cc:2258] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:05.839840 2516 catalog_manager.cc:4841] TS e7ed3c779c02428ba07bd9e83cebac05 (127.0.205.129:42927): tablet c23c8f1076554beb964f0e401c3c495e (table test-table [id=4894e248839b4aafa81b7a63587dddec]) successfully deleted
I20260808 02:57:05.841850 3405 ts_tablet_manager.cc:1905] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:05.849050 3405 ts_tablet_manager.cc:1918] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:57:05.849208 3405 log.cc:1196] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/c23c8f1076554beb964f0e401c3c495e
I20260808 02:57:05.849731 3405 ts_tablet_manager.cc:1939] T c23c8f1076554beb964f0e401c3c495e P c0fa833ff0dd41acac92136fc4cb8721: Deleting consensus metadata
I20260808 02:57:05.850934 2516 catalog_manager.cc:4841] TS c0fa833ff0dd41acac92136fc4cb8721 (127.0.205.131:43463): tablet c23c8f1076554beb964f0e401c3c495e (table test-table [id=4894e248839b4aafa81b7a63587dddec]) successfully deleted
Aug 08 02:57:05 dist-test-slave-6ntk krb5kdc[2472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157824, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:05.871438 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.841625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.0.205.129:42927)
0808 02:57:05.841794 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.841800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.850665 (+ 8865us) server_negotiation.cc:407] Connection header received
0808 02:57:05.851192 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.851198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.851295 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.851380 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:05.853811 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.854984 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.856894 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.857636 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.857777 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.863398 (+ 5621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:05.863418 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:05.863437 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:05.863491 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:05.867967 (+ 4476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:05.868913 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:05.868925 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:05.868930 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:05.869137 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:05.869603 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:05.869608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:05.869612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:05.870122 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:05.870467 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.870781 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.871035 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 02:57:05.872681 2845 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58350
I20260808 02:57:05.900938 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.876404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60234 (local address 127.0.205.190:40643)
0808 02:57:05.876978 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.876984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.883452 (+ 6468us) server_negotiation.cc:407] Connection header received
0808 02:57:05.883668 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.883675 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.883779 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.883911 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:05.885831 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.887115 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.888997 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.889772 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.889955 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.893135 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:05.893142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:05.893147 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:05.893210 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:05.897156 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:05.898125 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:05.898132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:05.898135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:05.898350 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:05.898892 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:05.898899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:05.898902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:05.899303 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:05.899572 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.899919 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.900196 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":266,"threads_started":1}
W20260808 02:57:05.901702 2550 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60234
I20260808 02:57:05.904587 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2659
I20260808 02:57:05.924634 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.906649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58354 (local address 127.0.205.129:42927)
0808 02:57:05.906941 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.906948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.907641 (+ 693us) server_negotiation.cc:407] Connection header received
0808 02:57:05.907798 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.907806 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.907903 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.908023 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:05.909649 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.910804 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.912673 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.913306 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.913443 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.916781 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:05.916797 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:05.916802 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:05.916848 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:05.920531 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:05.921363 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:05.921370 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:05.921374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:05.921553 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:05.922037 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:05.922042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:05.922046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:05.922519 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:05.923007 (+ 488us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.923795 (+ 788us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.924078 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
W20260808 02:57:05.926139 2845 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58354
W20260808 02:57:05.926846 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2659: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:57:05.927060 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 2659
I20260808 02:57:05.977397 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2860
Aug 08 02:57:05 dist-test-slave-6ntk krb5kdc[2472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157824, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:57:05.998978 3353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:05.978309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.0.205.130:36787)
0808 02:57:05.978643 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:05.978651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:05.978703 (+ 52us) server_negotiation.cc:407] Connection header received
0808 02:57:05.978836 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:05.978843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:05.978976 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:05.979104 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:05.980675 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.982065 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.983987 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:05.984736 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:05.984911 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:05.990729 (+ 5818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:05.990750 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:05.990772 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:05.990836 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:05.995330 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:05.996175 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:05.996185 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:05.996189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:05.996370 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:05.996800 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:05.996810 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:05.996813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:05.997339 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:05.998193 (+ 854us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:05.998357 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 02:57:05.998597 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
W20260808 02:57:05.999781 3055 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48442
W20260808 02:57:06.000593 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2860: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:57:06.000777 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 2860
I20260808 02:57:06.025746 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 3067
Aug 08 02:57:06 dist-test-slave-6ntk krb5kdc[2472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157824, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:57:06.043417 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:06.026650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33230 (local address 127.0.205.131:43463)
0808 02:57:06.026854 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:06.026860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:06.026882 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:57:06.027024 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:06.027029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:06.027109 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:06.027228 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:06.028506 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:06.029628 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:06.031042 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:06.031643 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:06.031785 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:06.036273 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:06.036287 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:06.036307 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:06.036351 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:06.040550 (+ 4199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:06.041251 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:06.041261 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:06.041265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:06.041412 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:06.041757 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:06.041762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:06.041765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:06.042267 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:06.042611 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:06.042819 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 02:57:06.043049 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 02:57:06.044167 3254 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33230
W20260808 02:57:06.044888 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 3067: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:57:06.045091 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 3067
I20260808 02:57:06.074815 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2493
I20260808 02:57:06.076045 2614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:06.199380 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 2493
2026-08-08T02:57:06Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5446 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:57:06 dist-test-slave-6ntk krb5kdc[3418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:57:06 dist-test-slave-6ntk krb5kdc[3418](info): set up 2 sockets
Aug 08 02:57:06 dist-test-slave-6ntk krb5kdc[3418](info): commencing operation
krb5kdc: starting...
W20260808 02:57:07.515591 822 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:57:07 dist-test-slave-6ntk krb5kdc[3418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:57:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:57:07.738184 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:41291
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:41291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:08.075665 3434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:08.075976 3434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:08.076133 3434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:08.083673 3434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:08.083787 3434 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:57:08.083856 3434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:08.083961 3434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:08.084045 3434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:08.084151 3434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:08.094295 3434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:41291
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:41291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:08.095690 3434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:08.097059 3434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:08.109716 3440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:08.109805 3441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:08.109759 3443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:08.110800 3434 server_base.cc:1034] running on GCE node
I20260808 02:57:08.111649 3434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:08.113157 3434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:08.114477 3434 hybrid_clock.cc:648] HybridClock initialized: now 1786157828114402 us; error 43 us; skew 500 ppm
I20260808 02:57:08.114878 3434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:08 dist-test-slave-6ntk krb5kdc[3418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157828, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:08.118603 3434 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:57:08.120362 3434 webserver.cc:460] Webserver started at http://127.0.205.190:44605/ using document root <none> and password file <none>
I20260808 02:57:08.121160 3434 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:08.121317 3434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:08.121719 3434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:08.124938 3434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "27f9e97edeb748c4bbfc4077dcd59d31"
format_stamp: "Formatted at 2026-08-08 02:57:08 on dist-test-slave-6ntk"
server_key: "146f12b869fcd6de2cda00a3c12bdbe0"
server_key_iv: "458167c08bbc1f42de0a75b8fc2fc1a7"
server_key_version: "encryptionkey@0"
I20260808 02:57:08.125761 3434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "27f9e97edeb748c4bbfc4077dcd59d31"
format_stamp: "Formatted at 2026-08-08 02:57:08 on dist-test-slave-6ntk"
server_key: "146f12b869fcd6de2cda00a3c12bdbe0"
server_key_iv: "458167c08bbc1f42de0a75b8fc2fc1a7"
server_key_version: "encryptionkey@0"
I20260808 02:57:08.131247 3434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:57:08.134881 3453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:08.136299 3434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:57:08.136507 3434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "27f9e97edeb748c4bbfc4077dcd59d31"
format_stamp: "Formatted at 2026-08-08 02:57:08 on dist-test-slave-6ntk"
server_key: "146f12b869fcd6de2cda00a3c12bdbe0"
server_key_iv: "458167c08bbc1f42de0a75b8fc2fc1a7"
server_key_version: "encryptionkey@0"
I20260808 02:57:08.136747 3434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:08.158435 3434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:08.162788 3434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:08.163097 3434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:08.185925 3434 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:41291
I20260808 02:57:08.185945 3546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:41291 every 8 connection(s)
I20260808 02:57:08.188066 3434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:08.190632 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 3434
I20260808 02:57:08.190881 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:08.191759 822 external_mini_cluster.cc:1428] Setting key 3e45389243d6fcf406f02a89eb01f1ca
I20260808 02:57:08.193526 3547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:08.205865 3547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31: Bootstrap starting.
Aug 08 02:57:08 dist-test-slave-6ntk krb5kdc[3418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:08.209852 3547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:08.211154 3547 log.cc:824] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:08.215164 3547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31: No bootstrap required, opened a new log
I20260808 02:57:08.216063 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:08.194031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.0.205.190:41291)
0808 02:57:08.194785 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:08.194797 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:08.194838 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:57:08.196441 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:08.196465 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:08.196857 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:08.197279 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:08.198843 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:08.200200 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:08.201455 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:08.202236 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:08.202444 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:08.207549 (+ 5105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:08.207560 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:08.207574 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:08.207616 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:08.211024 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:08.212021 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:08.212030 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:08.212039 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:08.212333 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:08.212886 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:08.212890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:08.212899 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:08.213502 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:08.213742 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:08.214385 (+ 643us) server_negotiation.cc:299] Negotiation successful
0808 02:57:08.214805 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":346,"threads_started":1}
I20260808 02:57:08.224395 3547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f9e97edeb748c4bbfc4077dcd59d31" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41291 } }
I20260808 02:57:08.224840 3547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:08.224985 3547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f9e97edeb748c4bbfc4077dcd59d31, State: Initialized, Role: FOLLOWER
I20260808 02:57:08.225647 3547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f9e97edeb748c4bbfc4077dcd59d31" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41291 } }
I20260808 02:57:08.225988 3547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:08.226119 3547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:08.226281 3547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:08.228632 3547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f9e97edeb748c4bbfc4077dcd59d31" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41291 } }
I20260808 02:57:08.229184 3547 leader_election.cc:302] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f9e97edeb748c4bbfc4077dcd59d31; no voters:
I20260808 02:57:08.229924 3547 leader_election.cc:288] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:08.230089 3554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:08.232034 3547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:08.232465 3554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [term 1 LEADER]: Becoming Leader. State: Replica: 27f9e97edeb748c4bbfc4077dcd59d31, State: Running, Role: LEADER
I20260808 02:57:08.233023 3554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f9e97edeb748c4bbfc4077dcd59d31" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41291 } }
I20260808 02:57:08.236133 3557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27f9e97edeb748c4bbfc4077dcd59d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f9e97edeb748c4bbfc4077dcd59d31" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41291 } } }
I20260808 02:57:08.236546 3557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:08.236982 3559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27f9e97edeb748c4bbfc4077dcd59d31. Latest consensus state: current_term: 1 leader_uuid: "27f9e97edeb748c4bbfc4077dcd59d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f9e97edeb748c4bbfc4077dcd59d31" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41291 } } }
I20260808 02:57:08.237282 3559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:08.243204 3570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:08.247786 3570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:08.256568 3570 catalog_manager.cc:1347] Generated new cluster ID: b8321bea3fa94796bc279a9486ccc69d
I20260808 02:57:08.256711 3570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:08.288010 3570 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:08.289438 3570 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:08.310797 3570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31: Generated new TSK 0
I20260808 02:57:08.311761 3570 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:57:08.365635 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:41291
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:08.708022 3584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:08.708305 3584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:08.708416 3584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:08.715803 3584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:08.715907 3584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:08.715992 3584 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:57:08.716044 3584 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:57:08.716115 3584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:08.725766 3584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.205.190:41291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:08.727095 3584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:08.728399 3584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:08.742236 3593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:08.742337 3590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:08.742292 3589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:08.747522 3584 server_base.cc:1034] running on GCE node
I20260808 02:57:08.748170 3584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:08.749186 3584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:08.750442 3584 hybrid_clock.cc:648] HybridClock initialized: now 1786157828750393 us; error 45 us; skew 500 ppm
I20260808 02:57:08.750779 3584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:08 dist-test-slave-6ntk krb5kdc[3418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157828, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:08.754120 3584 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:57:08.755749 3584 webserver.cc:460] Webserver started at http://127.0.205.129:43225/ using document root <none> and password file <none>
I20260808 02:57:08.756471 3584 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:08.756573 3584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:08.756839 3584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:08.759858 3584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "c52be984b29e4f99b1ff1d148b6fb368"
format_stamp: "Formatted at 2026-08-08 02:57:08 on dist-test-slave-6ntk"
server_key: "05201dd17e14d6981ee72454a1e9257b"
server_key_iv: "9c72a11b54643eb3720ed791000957c8"
server_key_version: "encryptionkey@0"
I20260808 02:57:08.760679 3584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "c52be984b29e4f99b1ff1d148b6fb368"
format_stamp: "Formatted at 2026-08-08 02:57:08 on dist-test-slave-6ntk"
server_key: "05201dd17e14d6981ee72454a1e9257b"
server_key_iv: "9c72a11b54643eb3720ed791000957c8"
server_key_version: "encryptionkey@0"
I20260808 02:57:08.766135 3584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:57:08.769502 3603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:08.770845 3584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:08.771014 3584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "c52be984b29e4f99b1ff1d148b6fb368"
format_stamp: "Formatted at 2026-08-08 02:57:08 on dist-test-slave-6ntk"
server_key: "05201dd17e14d6981ee72454a1e9257b"
server_key_iv: "9c72a11b54643eb3720ed791000957c8"
server_key_version: "encryptionkey@0"
I20260808 02:57:08.771232 3584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:08.792656 3584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:08.797379 3584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:08.797691 3584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:08.800637 3584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:08.800760 3584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:08.800894 3584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:08.800981 3584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:57:08 dist-test-slave-6ntk krb5kdc[3418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157828, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:08.824302 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:08.801115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:59907 (local address 127.0.205.190:41291)
0808 02:57:08.801989 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:08.801994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:08.802997 (+ 1003us) server_negotiation.cc:407] Connection header received
0808 02:57:08.805124 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:08.805130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:08.805228 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:08.805372 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:08.808252 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:08.809353 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:08.811766 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:08.812202 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:08.812345 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:08.817031 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:08.817037 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:08.817040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:08.817078 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:08.820396 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:08.821503 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:08.821512 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:08.821518 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:08.821702 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:08.822272 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:08.822279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:08.822284 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:08.822719 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:08.822889 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:08.823680 (+ 791us) server_negotiation.cc:299] Negotiation successful
0808 02:57:08.823924 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":618,"threads_started":1}
I20260808 02:57:08.825773 3618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:08.801836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41291 (local address 127.0.205.129:59907)
0808 02:57:08.802818 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:08.802857 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:08.804820 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:08.805844 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:08.805896 (+ 52us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:08.807014 (+ 1118us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:08.807986 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:08.808019 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:08.809532 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:08.809538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:08.811594 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:08.811607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:08.812482 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:08.812493 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:08.812720 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:08.813491 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:08.813525 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:08.816746 (+ 3221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:08.820552 (+ 3806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:08.820561 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:08.820588 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:08.821323 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:08.821840 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:08.821844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:08.821848 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:08.822126 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:08.822866 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:08.822874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:08.823281 (+ 407us) client_negotiation.cc:769] Sending connection context
0808 02:57:08.823581 (+ 300us) client_negotiation.cc:240] Negotiation successful
0808 02:57:08.824180 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":457,"threads_started":1}
I20260808 02:57:08.855494 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:08.838094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41291 (local address 127.0.205.129:56713)
0808 02:57:08.838772 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:08.838797 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:08.839021 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:08.839444 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:08.839449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:08.839968 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:08.840382 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:08.840393 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:08.841451 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:08.841458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:08.842899 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:08.842916 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:08.844415 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:08.844422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:08.844758 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:08.845822 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:08.845855 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:08.847417 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:08.852104 (+ 4687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:08.852113 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:08.852120 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:08.852949 (+ 829us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:08.853509 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:08.853514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:08.853518 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:08.853747 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:08.854573 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:08.854577 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:08.854744 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 02:57:08.854860 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 02:57:08.855132 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":361,"threads_started":1}
I20260808 02:57:08.855898 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:08.837934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:56713 (local address 127.0.205.190:41291)
0808 02:57:08.838184 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:08.838193 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:08.838937 (+ 744us) server_negotiation.cc:407] Connection header received
0808 02:57:08.839215 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:08.839221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:08.839301 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:08.839420 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:08.840530 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:08.841289 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:08.843183 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:08.843929 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:08.844091 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:08.847729 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:08.847738 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:08.847745 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:08.847796 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:08.851901 (+ 4105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:08.853148 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:08.853157 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:08.853162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:08.853369 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:08.853954 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:08.853961 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:08.853967 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:08.854438 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:08.854637 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:08.855060 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 02:57:08.855390 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:57:08.865808 3584 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:33851
I20260808 02:57:08.865847 3784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:33851 every 8 connection(s)
I20260808 02:57:08.867158 3584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:08.868088 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 3584
I20260808 02:57:08.868302 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:57:08.869118 822 external_mini_cluster.cc:1428] Setting key 2f0a37fb543efcb234cd0e7e8bc30f51
I20260808 02:57:08.869532 3785 heartbeater.cc:347] Connected to a master server at 127.0.205.190:41291
I20260808 02:57:08.869812 3785 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:08.870682 3785 heartbeater.cc:511] Master 127.0.205.190:41291 requested a full tablet report, sending...
I20260808 02:57:08.872793 3476 ts_manager.cc:194] Registered new tserver with Master: c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851)
I20260808 02:57:08.874409 3476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:59907
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:57:08.918438 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:41291
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:09.263123 3790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:09.263424 3790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:09.263528 3790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:09.270838 3790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:09.270946 3790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:09.271029 3790 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:57:09.271082 3790 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:57:09.271144 3790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:09.281203 3790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.205.190:41291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:09.282500 3790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:09.283874 3790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:09.297468 3796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:09.297520 3797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:09.297544 3800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:09.298347 3790 server_base.cc:1034] running on GCE node
I20260808 02:57:09.299103 3790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:09.300055 3790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:09.301297 3790 hybrid_clock.cc:648] HybridClock initialized: now 1786157829301257 us; error 55 us; skew 500 ppm
I20260808 02:57:09.301664 3790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:09 dist-test-slave-6ntk krb5kdc[3418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157829, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:09.304872 3790 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:57:09.306479 3790 webserver.cc:460] Webserver started at http://127.0.205.130:34387/ using document root <none> and password file <none>
I20260808 02:57:09.307183 3790 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:09.307336 3790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:09.307678 3790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:09.310818 3790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "688fcbfd26c442e086cecbe0b7234a95"
format_stamp: "Formatted at 2026-08-08 02:57:09 on dist-test-slave-6ntk"
server_key: "affdc809b9d94e3a351a2e3d69199dbf"
server_key_iv: "9e97f2e2395c4c637c339b5563601e4d"
server_key_version: "encryptionkey@0"
I20260808 02:57:09.311650 3790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "688fcbfd26c442e086cecbe0b7234a95"
format_stamp: "Formatted at 2026-08-08 02:57:09 on dist-test-slave-6ntk"
server_key: "affdc809b9d94e3a351a2e3d69199dbf"
server_key_iv: "9e97f2e2395c4c637c339b5563601e4d"
server_key_version: "encryptionkey@0"
I20260808 02:57:09.316854 3790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:57:09.320600 3807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:09.322011 3790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:09.322186 3790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "688fcbfd26c442e086cecbe0b7234a95"
format_stamp: "Formatted at 2026-08-08 02:57:09 on dist-test-slave-6ntk"
server_key: "affdc809b9d94e3a351a2e3d69199dbf"
server_key_iv: "9e97f2e2395c4c637c339b5563601e4d"
server_key_version: "encryptionkey@0"
I20260808 02:57:09.322413 3790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:09.346799 3790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:09.351158 3790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:09.351504 3790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:09.354321 3790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:09.354471 3790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:09.354604 3790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:09.354705 3790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:57:09 dist-test-slave-6ntk krb5kdc[3418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157829, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:09.378909 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:09.354988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:60341 (local address 127.0.205.190:41291)
0808 02:57:09.355228 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:09.355234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:09.356808 (+ 1574us) server_negotiation.cc:407] Connection header received
0808 02:57:09.359036 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:09.359042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:09.359142 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:09.359260 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:09.361579 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.362963 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.365542 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.366381 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.366539 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:09.371647 (+ 5108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:09.371653 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:09.371656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:09.371693 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:09.374682 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:09.375665 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:09.375673 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:09.375680 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:09.375853 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:09.376583 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:09.376591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:09.376596 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:09.376916 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:09.377090 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:09.378302 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0808 02:57:09.378530 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 02:57:09.380007 3823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:09.355697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41291 (local address 127.0.205.130:60341)
0808 02:57:09.356622 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:09.356671 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:09.358736 (+ 2065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:09.359543 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:09.359558 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:09.360376 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:09.361310 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:09.361342 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.363167 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.363175 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:09.365407 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:09.365424 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.366661 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.366671 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:09.366937 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:09.367763 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:09.367803 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:09.371389 (+ 3586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:09.374844 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:09.374859 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:09.374882 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:09.375522 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:09.376011 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:09.376022 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:09.376028 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:09.376436 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:09.377201 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:09.377216 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:09.377744 (+ 528us) client_negotiation.cc:769] Sending connection context
0808 02:57:09.378212 (+ 468us) client_negotiation.cc:240] Negotiation successful
0808 02:57:09.378801 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":377,"threads_started":1}
I20260808 02:57:09.410418 3790 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:40139
I20260808 02:57:09.410521 3998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:40139 every 8 connection(s)
I20260808 02:57:09.410689 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:09.392837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:57417 (local address 127.0.205.190:41291)
0808 02:57:09.393068 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:09.393077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:09.393789 (+ 712us) server_negotiation.cc:407] Connection header received
0808 02:57:09.394128 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:09.394135 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:09.394237 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:09.394380 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:09.395985 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.397538 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.399594 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.400172 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.400320 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:09.402796 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:09.402806 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:09.402812 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:09.402860 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:09.407133 (+ 4273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:09.408059 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:09.408066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:09.408069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:09.408243 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:09.408770 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:09.408775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:09.408778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:09.409171 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:09.409294 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:09.410038 (+ 744us) server_negotiation.cc:299] Negotiation successful
0808 02:57:09.410318 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 02:57:09.410987 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:09.392844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41291 (local address 127.0.205.130:57417)
0808 02:57:09.393629 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:09.393658 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:09.393930 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:09.394418 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:09.394423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:09.395301 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:09.395764 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:09.395784 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.397707 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.397711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:09.399427 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:09.399445 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.400410 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.400420 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:09.400603 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:09.401413 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:09.401447 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:09.402591 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:09.407314 (+ 4723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:09.407326 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:09.407329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:09.407896 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:09.408408 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:09.408416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:09.408423 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:09.408615 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:09.409305 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:09.409313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:09.409598 (+ 285us) client_negotiation.cc:769] Sending connection context
0808 02:57:09.409812 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 02:57:09.410417 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":372,"threads_started":1}
I20260808 02:57:09.412276 3790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:09.415151 3999 heartbeater.cc:347] Connected to a master server at 127.0.205.190:41291
I20260808 02:57:09.415405 3999 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:09.416090 3999 heartbeater.cc:511] Master 127.0.205.190:41291 requested a full tablet report, sending...
I20260808 02:57:09.417516 3476 ts_manager.cc:194] Registered new tserver with Master: 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130:40139)
I20260808 02:57:09.418430 3476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:60341
I20260808 02:57:09.421113 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 3790
I20260808 02:57:09.421303 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:57:09.421765 822 external_mini_cluster.cc:1428] Setting key 85d7e22393f364101f3004174333b795
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:57:09.469304 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:41291
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:09.818846 4005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:09.819150 4005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:09.819329 4005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:09.826851 4005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:09.826990 4005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:09.827149 4005 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:57:09.827240 4005 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:57:09.827351 4005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:09.837397 4005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.205.190:41291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:09.838693 4005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:09.839970 4005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:09.853801 4011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:09.853711 4014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:09.853776 4016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:09.854833 4005 server_base.cc:1034] running on GCE node
I20260808 02:57:09.855484 4005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:09.856498 4005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:09.857714 4005 hybrid_clock.cc:648] HybridClock initialized: now 1786157829857663 us; error 44 us; skew 500 ppm
I20260808 02:57:09.858070 4005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:09 dist-test-slave-6ntk krb5kdc[3418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157829, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:09.861398 4005 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:57:09.862990 4005 webserver.cc:460] Webserver started at http://127.0.205.131:35895/ using document root <none> and password file <none>
I20260808 02:57:09.863723 4005 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:09.863867 4005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:09.864210 4005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:09.867327 4005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "a5e45cdac1d045c38dc2791340f7f922"
format_stamp: "Formatted at 2026-08-08 02:57:09 on dist-test-slave-6ntk"
server_key: "714a0a07f1b6884a5064700bf4dd74c7"
server_key_iv: "0bb908ff3c059d5ea1406b5b9ba1a744"
server_key_version: "encryptionkey@0"
I20260808 02:57:09.868125 4005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "a5e45cdac1d045c38dc2791340f7f922"
format_stamp: "Formatted at 2026-08-08 02:57:09 on dist-test-slave-6ntk"
server_key: "714a0a07f1b6884a5064700bf4dd74c7"
server_key_iv: "0bb908ff3c059d5ea1406b5b9ba1a744"
server_key_version: "encryptionkey@0"
I20260808 02:57:09.873623 4005 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:57:09.877422 4026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:09.877452 3785 heartbeater.cc:503] Master 127.0.205.190:41291 was elected leader, sending a full tablet report...
I20260808 02:57:09.878979 4005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:09.879174 4005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "a5e45cdac1d045c38dc2791340f7f922"
format_stamp: "Formatted at 2026-08-08 02:57:09 on dist-test-slave-6ntk"
server_key: "714a0a07f1b6884a5064700bf4dd74c7"
server_key_iv: "0bb908ff3c059d5ea1406b5b9ba1a744"
server_key_version: "encryptionkey@0"
I20260808 02:57:09.879422 4005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:09.921943 4005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:09.926910 4005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:09.927268 4005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:09.930408 4005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:09.930567 4005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:09.930696 4005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:09.930814 4005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:57:09 dist-test-slave-6ntk krb5kdc[3418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157829, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:09.956207 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:09.930928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:37421 (local address 127.0.205.190:41291)
0808 02:57:09.931761 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:09.931767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:09.932865 (+ 1098us) server_negotiation.cc:407] Connection header received
0808 02:57:09.936182 (+ 3317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:09.936189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:09.936280 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:09.936443 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:09.938795 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.939541 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.942352 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.943345 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.943521 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:09.947813 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:09.947822 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:09.947830 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:09.947877 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:09.952134 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:09.952964 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:09.952973 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:09.952980 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:09.953176 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:09.953633 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:09.953641 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:09.953648 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:09.954139 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:09.954330 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:09.955343 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0808 02:57:09.955632 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":595,"threads_started":1}
I20260808 02:57:09.957330 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:09.931683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41291 (local address 127.0.205.131:37421)
0808 02:57:09.932658 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:09.932718 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:09.935763 (+ 3045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:09.936688 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:09.936700 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:09.937509 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:09.938529 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:09.938563 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.939703 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.939708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:09.942161 (+ 2453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:09.942186 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.943579 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.943584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:09.943761 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:09.944542 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:09.944564 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:09.947430 (+ 2866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:09.952312 (+ 4882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:09.952323 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:09.952346 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:09.952807 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:09.953302 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:09.953305 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:09.953308 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:09.953509 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:09.954316 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:09.954326 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:09.954787 (+ 461us) client_negotiation.cc:769] Sending connection context
0808 02:57:09.955191 (+ 404us) client_negotiation.cc:240] Negotiation successful
0808 02:57:09.955888 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":370,"threads_started":1}
I20260808 02:57:09.983580 4005 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:43729
I20260808 02:57:09.983618 4208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:43729 every 8 connection(s)
I20260808 02:57:09.985299 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:09.969107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:35179 (local address 127.0.205.190:41291)
0808 02:57:09.969340 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:09.969349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:09.970103 (+ 754us) server_negotiation.cc:407] Connection header received
0808 02:57:09.970427 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:09.970436 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:09.970527 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:09.970641 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:09.972205 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.973278 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.974906 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.975496 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.975647 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:09.978559 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:09.978564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:09.978567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:09.978597 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:09.982012 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:09.983003 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:09.983008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:09.983011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:09.983181 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:09.983726 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:09.983731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:09.983735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:09.984085 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:09.984224 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:09.984867 (+ 643us) server_negotiation.cc:299] Negotiation successful
0808 02:57:09.985043 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 02:57:09.985287 4005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:57:09.985565 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:09.969093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41291 (local address 127.0.205.131:35179)
0808 02:57:09.969966 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:09.969994 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:09.970177 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:09.970664 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:09.970672 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:09.971489 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:09.972054 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:09.972074 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.973404 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.973415 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:09.974746 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:09.974758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:09.975692 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:09.975700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:09.975909 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:09.976860 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:09.976903 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:09.978303 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:09.982143 (+ 3840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:09.982148 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:09.982153 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:09.982847 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:09.983354 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:09.983363 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:09.983370 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:09.983559 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:09.984347 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:09.984352 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:09.984558 (+ 206us) client_negotiation.cc:769] Sending connection context
0808 02:57:09.984719 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 02:57:09.985084 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":411,"threads_started":1}
I20260808 02:57:09.987010 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 4005
I20260808 02:57:09.987195 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:57:09.987682 822 external_mini_cluster.cc:1428] Setting key 5b60202ddb9ca2607a4e5a21def75eed
I20260808 02:57:09.989281 4210 heartbeater.cc:347] Connected to a master server at 127.0.205.190:41291
I20260808 02:57:09.989548 4210 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:09.990401 4210 heartbeater.cc:511] Master 127.0.205.190:41291 requested a full tablet report, sending...
I20260808 02:57:09.991698 3476 ts_manager.cc:194] Registered new tserver with Master: a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729)
I20260808 02:57:09.992487 3476 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:37421
I20260808 02:57:10.003511 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:57:10 dist-test-slave-6ntk krb5kdc[3418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:57:10 dist-test-slave-6ntk krb5kdc[3418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:10.056946 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.042955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60092 (local address 127.0.205.190:41291)
0808 02:57:10.043138 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.043143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.043538 (+ 395us) server_negotiation.cc:407] Connection header received
0808 02:57:10.043699 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.043704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.043788 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.043886 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:10.045289 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.046105 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.047349 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.048030 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.048158 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.052096 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:10.052102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:10.052105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:10.052139 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:10.054727 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:10.055340 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:10.055344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:10.055346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:10.055479 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:10.055804 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:10.055807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:10.055808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:10.056076 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:10.056274 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.056510 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.056671 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:57:10.061249 3476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:57:10.068207 3476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:10.093088 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.081715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:40139 (local address 127.0.0.1:35234)
0808 02:57:10.083398 (+ 1683us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.083449 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.083679 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.084893 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.084902 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.084938 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:10.085336 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.085347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.087357 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.087366 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.090121 (+ 2755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.090138 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.092055 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.092062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.092293 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.092359 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.092498 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.092706 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"thread_start_us":1018,"threads_started":1}
I20260808 02:57:10.094444 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.082052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35234 (local address 127.0.205.130:40139)
0808 02:57:10.082719 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.082729 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.084255 (+ 1526us) server_negotiation.cc:407] Connection header received
0808 02:57:10.084399 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.084405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.084589 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.085548 (+ 959us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:10.085703 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.087185 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.090325 (+ 3140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.091801 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.093608 (+ 1807us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.093821 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.093936 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.094124 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":346,"threads_started":1}
I20260808 02:57:10.094501 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.083062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52448 (local address 127.0.205.129:33851)
0808 02:57:10.084295 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.084303 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.086478 (+ 2175us) server_negotiation.cc:407] Connection header received
0808 02:57:10.086772 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.086779 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.087013 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.087172 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:10.088486 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.089977 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.091844 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.092991 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.093162 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.093396 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.093983 (+ 587us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.094159 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1089,"thread_start_us":958,"threads_started":1}
I20260808 02:57:10.095108 4232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.083046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:33851 (local address 127.0.0.1:52448)
0808 02:57:10.086362 (+ 3316us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.086382 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.086586 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.087434 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.087440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.087473 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:10.088288 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.088302 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.090155 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.090160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.091675 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.091688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.093543 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.093551 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.093732 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.093772 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.094713 (+ 941us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.094835 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3193,"mutex_wait_us":29,"thread_start_us":4378,"threads_started":1}
I20260808 02:57:10.097576 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.083689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43729 (local address 127.0.0.1:59840)
0808 02:57:10.088007 (+ 4318us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.088025 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.088196 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.088881 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.088886 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.088921 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:10.089290 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.089301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.091471 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.091477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.093698 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.093713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.096723 (+ 3010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.096730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.096927 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.096970 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.097097 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.097232 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4122,"thread_start_us":2412,"threads_started":1}
I20260808 02:57:10.097684 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.084431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59840 (local address 127.0.205.131:43729)
0808 02:57:10.085186 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.085195 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.088147 (+ 2952us) server_negotiation.cc:407] Connection header received
0808 02:57:10.088412 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.088421 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.088656 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.088862 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:10.089525 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.091273 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.093931 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.096187 (+ 2256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.096305 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.096529 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.097170 (+ 641us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.097335 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":350,"threads_started":1}
I20260808 02:57:10.098649 3886 tablet_service.cc:1467] Processing CreateTablet for tablet cd5171a3b22145c1a615a56233d11358 (DEFAULT_TABLE table=test-table [id=6b7a9863bc5146eeb53b5f7ffe60e7fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:10.098758 3683 tablet_service.cc:1467] Processing CreateTablet for tablet cd5171a3b22145c1a615a56233d11358 (DEFAULT_TABLE table=test-table [id=6b7a9863bc5146eeb53b5f7ffe60e7fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:10.100005 3886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd5171a3b22145c1a615a56233d11358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:10.100201 3683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd5171a3b22145c1a615a56233d11358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:10.101467 4110 tablet_service.cc:1467] Processing CreateTablet for tablet cd5171a3b22145c1a615a56233d11358 (DEFAULT_TABLE table=test-table [id=6b7a9863bc5146eeb53b5f7ffe60e7fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:10.103046 4110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd5171a3b22145c1a615a56233d11358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:10.113380 4235 tablet_bootstrap.cc:492] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: Bootstrap starting.
I20260808 02:57:10.115342 4239 tablet_bootstrap.cc:492] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: Bootstrap starting.
I20260808 02:57:10.117079 4235 tablet_bootstrap.cc:654] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:10.117939 4240 tablet_bootstrap.cc:492] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: Bootstrap starting.
I20260808 02:57:10.118566 4235 log.cc:824] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:10.120060 4239 tablet_bootstrap.cc:654] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:10.121212 4240 tablet_bootstrap.cc:654] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:10.121677 4239 log.cc:824] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:10.122282 4240 log.cc:824] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:10.125185 4239 tablet_bootstrap.cc:492] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: No bootstrap required, opened a new log
I20260808 02:57:10.125521 4239 ts_tablet_manager.cc:1397] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 02:57:10.125730 4235 tablet_bootstrap.cc:492] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: No bootstrap required, opened a new log
I20260808 02:57:10.126188 4235 ts_tablet_manager.cc:1397] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260808 02:57:10.126025 4240 tablet_bootstrap.cc:492] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: No bootstrap required, opened a new log
I20260808 02:57:10.126977 4240 ts_tablet_manager.cc:1397] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260808 02:57:10.134824 4240 raft_consensus.cc:348] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } }
I20260808 02:57:10.135406 4240 raft_consensus.cc:374] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:10.136085 4240 raft_consensus.cc:729] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e45cdac1d045c38dc2791340f7f922, State: Initialized, Role: FOLLOWER
I20260808 02:57:10.136785 4240 consensus_queue.cc:260] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } }
I20260808 02:57:10.136633 4239 raft_consensus.cc:348] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } }
I20260808 02:57:10.137073 4239 raft_consensus.cc:374] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:10.137184 4239 raft_consensus.cc:729] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52be984b29e4f99b1ff1d148b6fb368, State: Initialized, Role: FOLLOWER
I20260808 02:57:10.137374 4235 raft_consensus.cc:348] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } }
I20260808 02:57:10.137816 4235 raft_consensus.cc:374] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:10.137948 4235 raft_consensus.cc:729] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688fcbfd26c442e086cecbe0b7234a95, State: Initialized, Role: FOLLOWER
I20260808 02:57:10.138190 4210 heartbeater.cc:503] Master 127.0.205.190:41291 was elected leader, sending a full tablet report...
I20260808 02:57:10.138542 4235 consensus_queue.cc:260] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } }
I20260808 02:57:10.138752 4240 ts_tablet_manager.cc:1428] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260808 02:57:10.138984 4239 consensus_queue.cc:260] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } }
I20260808 02:57:10.144624 4239 ts_tablet_manager.cc:1428] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: Time spent starting tablet: real 0.019s user 0.015s sys 0.000s
I20260808 02:57:10.145623 3999 heartbeater.cc:503] Master 127.0.205.190:41291 was elected leader, sending a full tablet report...
I20260808 02:57:10.146368 4235 ts_tablet_manager.cc:1428] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: Time spent starting tablet: real 0.020s user 0.014s sys 0.005s
I20260808 02:57:10.162860 4247 raft_consensus.cc:482] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:10.163168 4247 raft_consensus.cc:504] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } }
I20260808 02:57:10.164865 4247 leader_election.cc:288] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729), c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851)
W20260808 02:57:10.165917 4000 tablet.cc:2369] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:57:10.172451 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.165414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:39191 (local address 127.0.205.129:33851)
0808 02:57:10.166011 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.166020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.166703 (+ 683us) server_negotiation.cc:407] Connection header received
0808 02:57:10.167030 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.167034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.167108 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.167222 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:10.167912 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.168650 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.170691 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.171436 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.171635 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.171720 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.172053 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.172181 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362}
I20260808 02:57:10.172645 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.165173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:33851 (local address 127.0.205.130:39191)
0808 02:57:10.166577 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.166603 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.166828 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.167240 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.167244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.167273 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:10.167747 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.167761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.168782 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.168790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.170524 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.170538 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.171687 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.171694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.171861 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.171913 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.172025 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.172161 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1238,"mutex_wait_us":60,"spinlock_wait_cycles":17664,"thread_start_us":1032,"threads_started":1}
I20260808 02:57:10.172684 4250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.165210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43729 (local address 127.0.205.130:40845)
0808 02:57:10.166863 (+ 1653us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.166881 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.167048 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.167392 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.167397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.167440 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:10.167871 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.167879 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.168928 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.168934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.170553 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.170566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.171718 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.171725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.171916 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.171939 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.172048 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.172178 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1521,"mutex_wait_us":29,"thread_start_us":1325,"threads_started":1}
I20260808 02:57:10.172466 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.165262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:40845 (local address 127.0.205.131:43729)
0808 02:57:10.165494 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.165503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.166952 (+ 1449us) server_negotiation.cc:407] Connection header received
0808 02:57:10.167201 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.167205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.167264 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.167361 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:10.168051 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.168779 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.170717 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.171475 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.171667 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.171748 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.172077 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.172202 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:57:10.173779 3715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd5171a3b22145c1a615a56233d11358" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52be984b29e4f99b1ff1d148b6fb368" is_pre_election: true
I20260808 02:57:10.173949 4142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd5171a3b22145c1a615a56233d11358" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e45cdac1d045c38dc2791340f7f922" is_pre_election: true
I20260808 02:57:10.174400 3715 raft_consensus.cc:2454] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 0.
I20260808 02:57:10.174522 4142 raft_consensus.cc:2454] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 0.
I20260808 02:57:10.175117 3815 leader_election.cc:302] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688fcbfd26c442e086cecbe0b7234a95, c52be984b29e4f99b1ff1d148b6fb368; no voters:
I20260808 02:57:10.175551 4247 raft_consensus.cc:2781] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:10.175706 4247 raft_consensus.cc:482] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:10.175827 4247 raft_consensus.cc:3037] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:10.178189 4247 raft_consensus.cc:504] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } }
I20260808 02:57:10.178818 4247 leader_election.cc:288] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 election: Requested vote from peers a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729), c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851)
I20260808 02:57:10.179172 4142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd5171a3b22145c1a615a56233d11358" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e45cdac1d045c38dc2791340f7f922"
I20260808 02:57:10.179391 4142 raft_consensus.cc:3037] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:10.179356 3715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd5171a3b22145c1a615a56233d11358" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52be984b29e4f99b1ff1d148b6fb368"
I20260808 02:57:10.179592 3715 raft_consensus.cc:3037] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:10.181671 4142 raft_consensus.cc:2454] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 1.
I20260808 02:57:10.181782 3715 raft_consensus.cc:2454] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 1.
I20260808 02:57:10.182220 3812 leader_election.cc:302] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688fcbfd26c442e086cecbe0b7234a95, a5e45cdac1d045c38dc2791340f7f922; no voters:
I20260808 02:57:10.182519 4247 raft_consensus.cc:2781] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:10.183000 4247 raft_consensus.cc:686] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 LEADER]: Becoming Leader. State: Replica: 688fcbfd26c442e086cecbe0b7234a95, State: Running, Role: LEADER
I20260808 02:57:10.183579 4247 consensus_queue.cc:237] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } }
I20260808 02:57:10.192804 3475 catalog_manager.cc:5495] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130). New cstate: current_term: 1 leader_uuid: "688fcbfd26c442e086cecbe0b7234a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } health_report { overall_health: UNKNOWN } } }
W20260808 02:57:10.237195 4211 tablet.cc:2369] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 02:57:10 dist-test-slave-6ntk krb5kdc[3418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157828, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:10.273984 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.260798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41291 (local address 127.0.0.1:60102)
0808 02:57:10.261407 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.261428 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.261565 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.261930 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.261941 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.262628 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:10.262909 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.262918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.263904 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.263908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.264860 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.264872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.265622 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.265626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.265734 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.266419 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:10.266436 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:10.268680 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:10.271659 (+ 2979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:10.271671 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:10.271677 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:10.272122 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:10.272457 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:10.272460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:10.272462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:10.272602 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:10.273069 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:10.273075 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:10.273232 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.273373 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.273590 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":321,"threads_started":1}
I20260808 02:57:10.274089 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.260734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.0.205.190:41291)
0808 02:57:10.260935 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.260942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.261522 (+ 580us) server_negotiation.cc:407] Connection header received
0808 02:57:10.261671 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.261675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.261736 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.261820 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:10.263043 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.263782 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.264988 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.265458 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.265552 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.268855 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:10.268861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:10.268864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:10.268894 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:10.271531 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:10.272224 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:10.272227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:10.272230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:10.272350 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:10.272695 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:10.272698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:10.272700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:10.272956 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:10.273067 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.273485 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.273706 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":68}
I20260808 02:57:10.279404 3475 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:60102:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:57:10.280542 3475 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:10.287762 3683 tablet_service.cc:1467] Processing CreateTablet for tablet 4f29b90c4f7c4a5a9326669af0dd6e13 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=74054784913d4badbcb39573654d1840]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:57:10.287844 3886 tablet_service.cc:1467] Processing CreateTablet for tablet 4f29b90c4f7c4a5a9326669af0dd6e13 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=74054784913d4badbcb39573654d1840]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:57:10.288246 4110 tablet_service.cc:1467] Processing CreateTablet for tablet 4f29b90c4f7c4a5a9326669af0dd6e13 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=74054784913d4badbcb39573654d1840]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:57:10.288753 3683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f29b90c4f7c4a5a9326669af0dd6e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:10.288827 3886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f29b90c4f7c4a5a9326669af0dd6e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:10.289048 4110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f29b90c4f7c4a5a9326669af0dd6e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:10.295210 4239 tablet_bootstrap.cc:492] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: Bootstrap starting.
I20260808 02:57:10.296315 4235 tablet_bootstrap.cc:492] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: Bootstrap starting.
I20260808 02:57:10.296401 4240 tablet_bootstrap.cc:492] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: Bootstrap starting.
I20260808 02:57:10.297017 4239 tablet_bootstrap.cc:654] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:10.298135 4240 tablet_bootstrap.cc:654] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:10.298341 4235 tablet_bootstrap.cc:654] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:10.300191 4239 tablet_bootstrap.cc:492] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: No bootstrap required, opened a new log
I20260808 02:57:10.300390 4239 ts_tablet_manager.cc:1397] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260808 02:57:10.301730 4240 tablet_bootstrap.cc:492] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: No bootstrap required, opened a new log
I20260808 02:57:10.301488 4239 raft_consensus.cc:348] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.301846 4239 raft_consensus.cc:374] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:10.302014 4239 raft_consensus.cc:729] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52be984b29e4f99b1ff1d148b6fb368, State: Initialized, Role: FOLLOWER
I20260808 02:57:10.302119 4240 ts_tablet_manager.cc:1397] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 02:57:10.302378 4239 consensus_queue.cc:260] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.303293 4239 ts_tablet_manager.cc:1428] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 02:57:10.303041 4248 tablet_replica.cc:439] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } }
I20260808 02:57:10.303473 4248 tablet_replica.cc:442] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:10.303646 4240 raft_consensus.cc:348] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.303917 4240 raft_consensus.cc:374] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:10.304023 4240 raft_consensus.cc:729] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e45cdac1d045c38dc2791340f7f922, State: Initialized, Role: FOLLOWER
I20260808 02:57:10.304255 4240 consensus_queue.cc:260] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.304899 4240 ts_tablet_manager.cc:1428] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 02:57:10.304831 4246 tablet_replica.cc:439] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } }
I20260808 02:57:10.305253 4246 tablet_replica.cc:442] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:10.305866 4235 tablet_bootstrap.cc:492] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: No bootstrap required, opened a new log
I20260808 02:57:10.306111 4235 ts_tablet_manager.cc:1397] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260808 02:57:10.307652 4235 raft_consensus.cc:348] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.307876 4235 raft_consensus.cc:374] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:10.307994 4235 raft_consensus.cc:729] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688fcbfd26c442e086cecbe0b7234a95, State: Initialized, Role: FOLLOWER
I20260808 02:57:10.308300 4235 consensus_queue.cc:260] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.308789 4247 tablet_replica.cc:439] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } }
I20260808 02:57:10.309054 4235 ts_tablet_manager.cc:1428] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 02:57:10.309080 4247 tablet_replica.cc:442] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 02:57:10.369858 3786 tablet.cc:2369] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:57:10.411814 4247 raft_consensus.cc:482] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:10.412058 4247 raft_consensus.cc:504] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.412845 4247 leader_election.cc:288] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851), a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729)
I20260808 02:57:10.413383 4142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f29b90c4f7c4a5a9326669af0dd6e13" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e45cdac1d045c38dc2791340f7f922" is_pre_election: true
I20260808 02:57:10.413408 3715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f29b90c4f7c4a5a9326669af0dd6e13" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52be984b29e4f99b1ff1d148b6fb368" is_pre_election: true
I20260808 02:57:10.413681 4142 raft_consensus.cc:2454] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 0.
I20260808 02:57:10.413712 3715 raft_consensus.cc:2454] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 0.
I20260808 02:57:10.414148 3815 leader_election.cc:302] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688fcbfd26c442e086cecbe0b7234a95, c52be984b29e4f99b1ff1d148b6fb368; no voters:
I20260808 02:57:10.414489 4247 raft_consensus.cc:2781] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:10.414597 4247 raft_consensus.cc:482] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:10.414696 4247 raft_consensus.cc:3037] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:10.417752 4247 raft_consensus.cc:504] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.418452 4247 leader_election.cc:288] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 election: Requested vote from peers c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851), a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729)
I20260808 02:57:10.419018 3715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f29b90c4f7c4a5a9326669af0dd6e13" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52be984b29e4f99b1ff1d148b6fb368"
I20260808 02:57:10.419021 4142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f29b90c4f7c4a5a9326669af0dd6e13" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e45cdac1d045c38dc2791340f7f922"
I20260808 02:57:10.419271 4142 raft_consensus.cc:3037] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:10.419291 3715 raft_consensus.cc:3037] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:10.421552 3715 raft_consensus.cc:2454] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 1.
I20260808 02:57:10.422128 3815 leader_election.cc:302] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688fcbfd26c442e086cecbe0b7234a95, c52be984b29e4f99b1ff1d148b6fb368; no voters:
I20260808 02:57:10.422308 4142 raft_consensus.cc:2454] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 1.
I20260808 02:57:10.422470 4247 raft_consensus.cc:2781] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:10.422708 4247 raft_consensus.cc:686] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 LEADER]: Becoming Leader. State: Replica: 688fcbfd26c442e086cecbe0b7234a95, State: Running, Role: LEADER
I20260808 02:57:10.422983 4247 consensus_queue.cc:237] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.424834 4251 tablet_replica.cc:439] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: TxnStatusTablet state changed. Reason: New leader 688fcbfd26c442e086cecbe0b7234a95. Latest consensus state: current_term: 1 leader_uuid: "688fcbfd26c442e086cecbe0b7234a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } }
I20260808 02:57:10.425262 4251 tablet_replica.cc:442] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:57:10.426592 3474 catalog_manager.cc:5495] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130). New cstate: current_term: 1 leader_uuid: "688fcbfd26c442e086cecbe0b7234a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } health_report { overall_health: UNKNOWN } } }
I20260808 02:57:10.430416 4268 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:57:10.430778 4268 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:57:10.457366 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.450633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:40139 (local address 127.0.0.1:35238)
0808 02:57:10.450903 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.450922 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.451044 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.451569 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.451572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.451582 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:57:10.451746 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.451754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.453216 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.453219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.454298 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.454310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.455371 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.455376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.455547 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.455675 (+ 128us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:57:10.456813 (+ 1138us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:57:10.456836 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.456928 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.457068 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 02:57:10.457391 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.450609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35238 (local address 127.0.205.130:40139)
0808 02:57:10.450840 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.450846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.450998 (+ 152us) server_negotiation.cc:407] Connection header received
0808 02:57:10.451247 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.451254 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.451383 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.451548 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:10.451888 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.453077 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.454440 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.455137 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.455306 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.455938 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:10.456680 (+ 742us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:10.456830 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.456972 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.457064 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:57:10.467312 4142 raft_consensus.cc:1261] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Refusing update from remote peer 688fcbfd26c442e086cecbe0b7234a95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:10.467319 3715 raft_consensus.cc:1261] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Refusing update from remote peer 688fcbfd26c442e086cecbe0b7234a95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:10.468466 4251 consensus_queue.cc:1035] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:10.468883 4247 consensus_queue.cc:1035] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:10.478551 4246 tablet_replica.cc:439] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: TxnStatusTablet state changed. Reason: New leader 688fcbfd26c442e086cecbe0b7234a95. Latest consensus state: current_term: 1 leader_uuid: "688fcbfd26c442e086cecbe0b7234a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } }
I20260808 02:57:10.479503 4246 tablet_replica.cc:442] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:10.481281 4272 mvcc.cc:205] Tried to move back new op lower bound from 7316102473581371392 to 7316102473415151616. Current Snapshot: MvccSnapshot[applied={T|T < 7316102473581371392}]
I20260808 02:57:10.482415 4246 tablet_replica.cc:439] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "688fcbfd26c442e086cecbe0b7234a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } }
I20260808 02:57:10.482685 4246 tablet_replica.cc:442] T 4f29b90c4f7c4a5a9326669af0dd6e13 P a5e45cdac1d045c38dc2791340f7f922: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:10.484642 4247 tablet_replica.cc:439] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "688fcbfd26c442e086cecbe0b7234a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } }
I20260808 02:57:10.485047 4247 tablet_replica.cc:442] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:57:10.494364 4248 tablet_replica.cc:439] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: TxnStatusTablet state changed. Reason: New leader 688fcbfd26c442e086cecbe0b7234a95. Latest consensus state: current_term: 1 leader_uuid: "688fcbfd26c442e086cecbe0b7234a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } }
I20260808 02:57:10.494858 4248 tablet_replica.cc:442] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:10.496706 4251 tablet_replica.cc:439] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "688fcbfd26c442e086cecbe0b7234a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } }
I20260808 02:57:10.497043 4251 tablet_replica.cc:442] T 4f29b90c4f7c4a5a9326669af0dd6e13 P 688fcbfd26c442e086cecbe0b7234a95: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:57:10.501375 4248 tablet_replica.cc:439] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "688fcbfd26c442e086cecbe0b7234a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } } }
I20260808 02:57:10.501657 4248 tablet_replica.cc:442] T 4f29b90c4f7c4a5a9326669af0dd6e13 P c52be984b29e4f99b1ff1d148b6fb368: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:57:10.510131 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.503590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.0.205.130:40139)
0808 02:57:10.503780 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.503786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.503992 (+ 206us) server_negotiation.cc:407] Connection header received
0808 02:57:10.504241 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.504247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.504332 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.504419 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:10.505020 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.506249 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.508002 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.508685 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.508820 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.509264 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:10.509424 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:10.509519 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.509674 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.509769 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:57:10.528587 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.522283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:35809 (local address 127.0.205.130:40139)
0808 02:57:10.522454 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.522459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.523226 (+ 767us) server_negotiation.cc:407] Connection header received
0808 02:57:10.523489 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.523492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.523549 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.523645 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:10.524171 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.524932 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.526615 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.527076 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.527229 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.527704 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:10.527870 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:10.528026 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.528072 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.528200 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":49}
I20260808 02:57:10.528702 4288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.522456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:40139 (local address 127.0.205.130:35809)
0808 02:57:10.523124 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.523140 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.523322 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.523659 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.523664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.523674 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:57:10.524028 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.524037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.525072 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.525077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.526474 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.526486 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.527287 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.527297 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.527435 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.527493 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:57:10.527973 (+ 480us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:57:10.527984 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.528058 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.528223 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":3,"thread_start_us":377,"threads_started":1}
I20260808 02:57:10.538651 3715 raft_consensus.cc:1261] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Refusing update from remote peer 688fcbfd26c442e086cecbe0b7234a95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:10.538802 4142 raft_consensus.cc:1261] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Refusing update from remote peer 688fcbfd26c442e086cecbe0b7234a95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:10.539259 4247 consensus_queue.cc:1035] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:10.539634 4284 consensus_queue.cc:1035] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:10.554648 4274 mvcc.cc:205] Tried to move back new op lower bound from 7316102473881468928 to 7316102472435834880. Current Snapshot: MvccSnapshot[applied={T|T < 7316102473881468928}]
I20260808 02:57:10.608054 3476 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:10.608366 3476 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:10.612020 3476 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31: Sending DeleteTablet for 3 replicas of tablet cd5171a3b22145c1a615a56233d11358
I20260808 02:57:10.613315 4110 tablet_service.cc:1514] Processing DeleteTablet for tablet cd5171a3b22145c1a615a56233d11358 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:10 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:59840
I20260808 02:57:10.613404 3683 tablet_service.cc:1514] Processing DeleteTablet for tablet cd5171a3b22145c1a615a56233d11358 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:10 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:52448
I20260808 02:57:10.614089 3885 tablet_service.cc:1514] Processing DeleteTablet for tablet cd5171a3b22145c1a615a56233d11358 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:10 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:35234
I20260808 02:57:10.616449 3476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:57:10.617599 3476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:10.622063 4297 tablet_replica.cc:330] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: stopping tablet replica
I20260808 02:57:10.622593 4297 raft_consensus.cc:2229] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:57:10.622849 4295 tablet_replica.cc:330] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: stopping tablet replica
I20260808 02:57:10.623363 4295 raft_consensus.cc:2229] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:10.623834 4295 raft_consensus.cc:2258] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:10.627327 4296 tablet_replica.cc:330] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: stopping tablet replica
I20260808 02:57:10.627794 4296 raft_consensus.cc:2229] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:10.628288 4296 raft_consensus.cc:2258] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:10.628964 3683 tablet_service.cc:1467] Processing CreateTablet for tablet 8c8b3421e13d40fe959dbcd53cc0ae42 (DEFAULT_TABLE table=test-table [id=3980325da1e4475a97366f83c753fdbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:10.629295 3885 tablet_service.cc:1467] Processing CreateTablet for tablet 8c8b3421e13d40fe959dbcd53cc0ae42 (DEFAULT_TABLE table=test-table [id=3980325da1e4475a97366f83c753fdbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:10.629647 3683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c8b3421e13d40fe959dbcd53cc0ae42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:10.629846 3885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c8b3421e13d40fe959dbcd53cc0ae42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:10.630039 4297 raft_consensus.cc:2258] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:10.630532 4296 ts_tablet_manager.cc:1905] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:10.631618 4295 ts_tablet_manager.cc:1905] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:10.631944 4297 ts_tablet_manager.cc:1905] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:10.638537 4235 tablet_bootstrap.cc:492] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95: Bootstrap starting.
I20260808 02:57:10.638554 4296 ts_tablet_manager.cc:1918] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:57:10.638720 4296 log.cc:1196] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/cd5171a3b22145c1a615a56233d11358
I20260808 02:57:10.639382 4296 ts_tablet_manager.cc:1939] T cd5171a3b22145c1a615a56233d11358 P c52be984b29e4f99b1ff1d148b6fb368: Deleting consensus metadata
I20260808 02:57:10.640563 4235 tablet_bootstrap.cc:654] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:10.641507 3456 catalog_manager.cc:4841] TS c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851): tablet cd5171a3b22145c1a615a56233d11358 (table test-table [id=6b7a9863bc5146eeb53b5f7ffe60e7fb]) successfully deleted
I20260808 02:57:10.643628 4297 ts_tablet_manager.cc:1918] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:57:10.643803 4297 log.cc:1196] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/cd5171a3b22145c1a615a56233d11358
I20260808 02:57:10.644416 4297 ts_tablet_manager.cc:1939] T cd5171a3b22145c1a615a56233d11358 P 688fcbfd26c442e086cecbe0b7234a95: Deleting consensus metadata
I20260808 02:57:10.645020 4110 tablet_service.cc:1467] Processing CreateTablet for tablet 8c8b3421e13d40fe959dbcd53cc0ae42 (DEFAULT_TABLE table=test-table [id=3980325da1e4475a97366f83c753fdbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:10.645524 4110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c8b3421e13d40fe959dbcd53cc0ae42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:10.646230 3454 catalog_manager.cc:4841] TS 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130:40139): tablet cd5171a3b22145c1a615a56233d11358 (table test-table [id=6b7a9863bc5146eeb53b5f7ffe60e7fb]) successfully deleted
I20260808 02:57:10.646766 4239 tablet_bootstrap.cc:492] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368: Bootstrap starting.
I20260808 02:57:10.648463 4239 tablet_bootstrap.cc:654] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:10.652021 4235 tablet_bootstrap.cc:492] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95: No bootstrap required, opened a new log
I20260808 02:57:10.652201 4235 ts_tablet_manager.cc:1397] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.009s
I20260808 02:57:10.652305 4240 tablet_bootstrap.cc:492] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922: Bootstrap starting.
I20260808 02:57:10.653023 4235 raft_consensus.cc:348] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.653295 4235 raft_consensus.cc:374] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:10.653383 4235 raft_consensus.cc:729] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688fcbfd26c442e086cecbe0b7234a95, State: Initialized, Role: FOLLOWER
I20260808 02:57:10.653712 4235 consensus_queue.cc:260] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.654340 4235 ts_tablet_manager.cc:1428] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:57:10.654409 4240 tablet_bootstrap.cc:654] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:10.660277 4240 tablet_bootstrap.cc:492] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922: No bootstrap required, opened a new log
I20260808 02:57:10.660512 4240 ts_tablet_manager.cc:1397] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 02:57:10.661088 4239 tablet_bootstrap.cc:492] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368: No bootstrap required, opened a new log
I20260808 02:57:10.661259 4239 ts_tablet_manager.cc:1397] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.002s
I20260808 02:57:10.661592 4240 raft_consensus.cc:348] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.661844 4240 raft_consensus.cc:374] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:10.661970 4240 raft_consensus.cc:729] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e45cdac1d045c38dc2791340f7f922, State: Initialized, Role: FOLLOWER
I20260808 02:57:10.661964 4239 raft_consensus.cc:348] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.662216 4239 raft_consensus.cc:374] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:10.662297 4239 raft_consensus.cc:729] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52be984b29e4f99b1ff1d148b6fb368, State: Initialized, Role: FOLLOWER
I20260808 02:57:10.662266 4240 consensus_queue.cc:260] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.662537 4239 consensus_queue.cc:260] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.663110 4239 ts_tablet_manager.cc:1428] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:57:10.663524 4240 ts_tablet_manager.cc:1428] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:57:10.660286 4295 ts_tablet_manager.cc:1918] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:57:10.663789 4295 log.cc:1196] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/cd5171a3b22145c1a615a56233d11358
I20260808 02:57:10.664391 4295 ts_tablet_manager.cc:1939] T cd5171a3b22145c1a615a56233d11358 P a5e45cdac1d045c38dc2791340f7f922: Deleting consensus metadata
I20260808 02:57:10.666453 3454 catalog_manager.cc:4841] TS a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729): tablet cd5171a3b22145c1a615a56233d11358 (table test-table [id=6b7a9863bc5146eeb53b5f7ffe60e7fb]) successfully deleted
I20260808 02:57:10.681078 4248 raft_consensus.cc:482] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:10.681288 4248 raft_consensus.cc:504] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.682735 4248 leader_election.cc:288] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130:40139), a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729)
I20260808 02:57:10.691540 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.683789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43729 (local address 127.0.205.129:54591)
0808 02:57:10.684441 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.684463 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.684596 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.685183 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.685188 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.685214 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:10.685666 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.685678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.687087 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.687092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.688992 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.689008 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.690651 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.690657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.690850 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.690887 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.690987 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.691129 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":264,"threads_started":1}
I20260808 02:57:10.691628 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.683892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:54591 (local address 127.0.205.131:43729)
0808 02:57:10.684698 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.684705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.684726 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:57:10.684797 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.684803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.684903 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.685007 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:10.685864 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.686945 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.689156 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.690471 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.690637 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.690720 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.691171 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.691314 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":574,"threads_started":1}
I20260808 02:57:10.691849 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.683149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:53187 (local address 127.0.205.130:40139)
0808 02:57:10.683390 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.683399 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.683729 (+ 330us) server_negotiation.cc:407] Connection header received
0808 02:57:10.684807 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.684811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.684900 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.685003 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:10.685531 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.686630 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.689291 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.690801 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.690993 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.691077 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.691442 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.691574 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 02:57:10.692250 4141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c8b3421e13d40fe959dbcd53cc0ae42" candidate_uuid: "c52be984b29e4f99b1ff1d148b6fb368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e45cdac1d045c38dc2791340f7f922" is_pre_election: true
I20260808 02:57:10.692509 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.683108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:40139 (local address 127.0.205.129:53187)
0808 02:57:10.683615 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.683638 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.684613 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.685021 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.685026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.685049 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:10.685352 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.685361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.686794 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.686802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.689100 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.689117 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.691044 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.691050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.691232 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.691273 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.692025 (+ 752us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.692162 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":269,"threads_started":1}
I20260808 02:57:10.692566 4141 raft_consensus.cc:2454] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c52be984b29e4f99b1ff1d148b6fb368 in term 0.
I20260808 02:57:10.693044 3604 leader_election.cc:302] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5e45cdac1d045c38dc2791340f7f922, c52be984b29e4f99b1ff1d148b6fb368; no voters:
I20260808 02:57:10.693503 4248 raft_consensus.cc:2781] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:10.693634 4248 raft_consensus.cc:482] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:10.693459 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c8b3421e13d40fe959dbcd53cc0ae42" candidate_uuid: "c52be984b29e4f99b1ff1d148b6fb368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688fcbfd26c442e086cecbe0b7234a95" is_pre_election: true
I20260808 02:57:10.693760 4248 raft_consensus.cc:3037] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:10.693946 3922 raft_consensus.cc:2454] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c52be984b29e4f99b1ff1d148b6fb368 in term 0.
I20260808 02:57:10.696269 4248 raft_consensus.cc:504] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.696851 4248 leader_election.cc:288] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [CANDIDATE]: Term 1 election: Requested vote from peers 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130:40139), a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729)
I20260808 02:57:10.697331 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c8b3421e13d40fe959dbcd53cc0ae42" candidate_uuid: "c52be984b29e4f99b1ff1d148b6fb368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688fcbfd26c442e086cecbe0b7234a95"
I20260808 02:57:10.697422 4141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c8b3421e13d40fe959dbcd53cc0ae42" candidate_uuid: "c52be984b29e4f99b1ff1d148b6fb368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e45cdac1d045c38dc2791340f7f922"
I20260808 02:57:10.697543 3922 raft_consensus.cc:3037] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:10.697621 4141 raft_consensus.cc:3037] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:10.699767 3922 raft_consensus.cc:2454] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c52be984b29e4f99b1ff1d148b6fb368 in term 1.
I20260808 02:57:10.700158 3604 leader_election.cc:302] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688fcbfd26c442e086cecbe0b7234a95, c52be984b29e4f99b1ff1d148b6fb368; no voters:
I20260808 02:57:10.700460 4248 raft_consensus.cc:2781] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:10.700527 4141 raft_consensus.cc:2454] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c52be984b29e4f99b1ff1d148b6fb368 in term 1.
I20260808 02:57:10.706225 4248 raft_consensus.cc:686] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 LEADER]: Becoming Leader. State: Replica: c52be984b29e4f99b1ff1d148b6fb368, State: Running, Role: LEADER
I20260808 02:57:10.706636 4248 consensus_queue.cc:237] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:10.710444 3476 catalog_manager.cc:5495] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 reported cstate change: term changed from 0 to 1, leader changed from <none> to c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129). New cstate: current_term: 1 leader_uuid: "c52be984b29e4f99b1ff1d148b6fb368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } health_report { overall_health: UNKNOWN } } }
I20260808 02:57:10.737172 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.730856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52452 (local address 127.0.205.129:33851)
0808 02:57:10.731460 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.731466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.731490 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:57:10.731592 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.731597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.731691 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.731995 (+ 304us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:10.732466 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.733363 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.734848 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.735262 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.735410 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.735931 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:10.736502 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:10.736662 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.736745 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.736824 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":668,"threads_started":1}
I20260808 02:57:10.754029 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.747846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:40139 (local address 127.0.205.129:51775)
0808 02:57:10.748471 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.748488 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.748606 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.748963 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.748966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.748975 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:57:10.749238 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.749245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.750770 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.750774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.751815 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.751827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.752843 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.752847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.752972 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.753024 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:57:10.753492 (+ 468us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:57:10.753500 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.753560 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.753656 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":333,"threads_started":1}
I20260808 02:57:10.754053 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.747798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:51775 (local address 127.0.205.130:40139)
0808 02:57:10.747981 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.747986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.748584 (+ 598us) server_negotiation.cc:407] Connection header received
0808 02:57:10.748732 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.748737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.748828 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.748947 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:10.749397 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.750646 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.751966 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.752666 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.752807 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.753211 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:10.753392 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:10.753487 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.753615 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.753699 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:57:10.763242 4141 raft_consensus.cc:1261] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Refusing update from remote peer c52be984b29e4f99b1ff1d148b6fb368: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:10.763583 3922 raft_consensus.cc:1261] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Refusing update from remote peer c52be984b29e4f99b1ff1d148b6fb368: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:10.764423 4307 consensus_queue.cc:1035] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:10.765061 4248 consensus_queue.cc:1035] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:10.799497 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.793601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:49313 (local address 127.0.205.129:33851)
0808 02:57:10.793763 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:10.793770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:10.793789 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:57:10.793853 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:10.793859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:10.793970 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:10.794082 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:10.794482 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.795613 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.797254 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.797779 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.797959 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.798510 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:10.798686 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:10.798788 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:10.799151 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 02:57:10.799238 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:57:10.799479 4288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:10.793364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:33851 (local address 127.0.205.130:49313)
0808 02:57:10.793567 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:10.793588 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:10.793704 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:10.794103 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:10.794109 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:10.794122 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:57:10.794326 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.794340 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.795752 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.795760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.797056 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:10.797069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:10.798027 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:10.798033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:10.798258 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:10.798309 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:57:10.798852 (+ 543us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:57:10.798864 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 02:57:10.798950 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:57:10.799116 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 02:57:10.800999 4274 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 02:57:12.416481 3476 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:12.416765 3476 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:12.421398 3476 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31: Sending DeleteTablet for 3 replicas of tablet 8c8b3421e13d40fe959dbcd53cc0ae42
I20260808 02:57:12.422506 3683 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c8b3421e13d40fe959dbcd53cc0ae42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:12 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:52448
I20260808 02:57:12.422565 4110 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c8b3421e13d40fe959dbcd53cc0ae42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:12 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:59840
I20260808 02:57:12.426131 3885 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c8b3421e13d40fe959dbcd53cc0ae42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:12 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:35234
I20260808 02:57:12.430027 4356 tablet_replica.cc:330] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95: stopping tablet replica
I20260808 02:57:12.430279 4356 raft_consensus.cc:2229] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:12.430564 4356 raft_consensus.cc:2258] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:12.433565 4356 ts_tablet_manager.cc:1905] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:12.435593 4354 tablet_replica.cc:330] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368: stopping tablet replica
I20260808 02:57:12.436094 4354 raft_consensus.cc:2229] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:57:12.436141 4355 tablet_replica.cc:330] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922: stopping tablet replica
I20260808 02:57:12.436393 4355 raft_consensus.cc:2229] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:12.436703 4355 raft_consensus.cc:2258] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:12.436863 4354 raft_consensus.cc:2258] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:12.437832 4355 ts_tablet_manager.cc:1905] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:12.438023 4354 ts_tablet_manager.cc:1905] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:12.444473 4355 ts_tablet_manager.cc:1918] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:57:12.444550 4356 ts_tablet_manager.cc:1918] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:57:12.444644 4355 log.cc:1196] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/8c8b3421e13d40fe959dbcd53cc0ae42
I20260808 02:57:12.444770 4356 log.cc:1196] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/8c8b3421e13d40fe959dbcd53cc0ae42
I20260808 02:57:12.445083 4355 ts_tablet_manager.cc:1939] T 8c8b3421e13d40fe959dbcd53cc0ae42 P a5e45cdac1d045c38dc2791340f7f922: Deleting consensus metadata
I20260808 02:57:12.445374 4354 ts_tablet_manager.cc:1918] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:57:12.445374 4356 ts_tablet_manager.cc:1939] T 8c8b3421e13d40fe959dbcd53cc0ae42 P 688fcbfd26c442e086cecbe0b7234a95: Deleting consensus metadata
I20260808 02:57:12.445566 4354 log.cc:1196] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/8c8b3421e13d40fe959dbcd53cc0ae42
I20260808 02:57:12.446170 4354 ts_tablet_manager.cc:1939] T 8c8b3421e13d40fe959dbcd53cc0ae42 P c52be984b29e4f99b1ff1d148b6fb368: Deleting consensus metadata
I20260808 02:57:12.446435 3454 catalog_manager.cc:4841] TS a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729): tablet 8c8b3421e13d40fe959dbcd53cc0ae42 (table test-table [id=3980325da1e4475a97366f83c753fdbf]) successfully deleted
I20260808 02:57:12.446821 3454 catalog_manager.cc:4841] TS 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130:40139): tablet 8c8b3421e13d40fe959dbcd53cc0ae42 (table test-table [id=3980325da1e4475a97366f83c753fdbf]) successfully deleted
I20260808 02:57:12.447328 3456 catalog_manager.cc:4841] TS c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851): tablet 8c8b3421e13d40fe959dbcd53cc0ae42 (table test-table [id=3980325da1e4475a97366f83c753fdbf]) successfully deleted
Aug 08 02:57:12 dist-test-slave-6ntk krb5kdc[3418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:57:12 dist-test-slave-6ntk krb5kdc[3418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:12.497422 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.474552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.0.205.190:41291)
0808 02:57:12.475180 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:12.475189 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:12.475215 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:57:12.475510 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:12.475518 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:12.475598 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:12.475722 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:12.477123 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.477998 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.479274 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.479692 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.479834 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.491544 (+ 11710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:12.491550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:12.491553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:12.491595 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:12.494651 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:12.495346 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:12.495354 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:12.495362 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:12.495496 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:12.495902 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:12.495913 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:12.495915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:12.496202 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:12.496474 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:12.496777 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 02:57:12.497041 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":304,"threads_started":1}
I20260808 02:57:12.501046 3476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:57:12.501621 3476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:12.508517 3683 tablet_service.cc:1467] Processing CreateTablet for tablet 0144ae57fd544c9fb70d3f1eb38af67e (DEFAULT_TABLE table=test-table [id=6ae70dcdef3b4ef38d79cfef012db8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:12.508598 3885 tablet_service.cc:1467] Processing CreateTablet for tablet 0144ae57fd544c9fb70d3f1eb38af67e (DEFAULT_TABLE table=test-table [id=6ae70dcdef3b4ef38d79cfef012db8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:12.509032 3683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0144ae57fd544c9fb70d3f1eb38af67e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:12.509169 3885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0144ae57fd544c9fb70d3f1eb38af67e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:12.509064 4110 tablet_service.cc:1467] Processing CreateTablet for tablet 0144ae57fd544c9fb70d3f1eb38af67e (DEFAULT_TABLE table=test-table [id=6ae70dcdef3b4ef38d79cfef012db8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:12.509476 4110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0144ae57fd544c9fb70d3f1eb38af67e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:12.515497 4366 tablet_bootstrap.cc:492] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368: Bootstrap starting.
I20260808 02:57:12.516579 4367 tablet_bootstrap.cc:492] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95: Bootstrap starting.
I20260808 02:57:12.517382 4366 tablet_bootstrap.cc:654] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:12.517508 4368 tablet_bootstrap.cc:492] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922: Bootstrap starting.
I20260808 02:57:12.518859 4367 tablet_bootstrap.cc:654] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:12.519349 4368 tablet_bootstrap.cc:654] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:12.520521 4366 tablet_bootstrap.cc:492] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368: No bootstrap required, opened a new log
I20260808 02:57:12.520700 4366 ts_tablet_manager.cc:1397] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 02:57:12.521461 4367 tablet_bootstrap.cc:492] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95: No bootstrap required, opened a new log
I20260808 02:57:12.521624 4367 ts_tablet_manager.cc:1397] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 02:57:12.521610 4366 raft_consensus.cc:348] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.522135 4366 raft_consensus.cc:374] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:12.522238 4366 raft_consensus.cc:729] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52be984b29e4f99b1ff1d148b6fb368, State: Initialized, Role: FOLLOWER
I20260808 02:57:12.522569 4368 tablet_bootstrap.cc:492] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922: No bootstrap required, opened a new log
I20260808 02:57:12.522703 4368 ts_tablet_manager.cc:1397] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 02:57:12.522537 4366 consensus_queue.cc:260] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.522569 4367 raft_consensus.cc:348] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.522846 4367 raft_consensus.cc:374] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:12.522989 4367 raft_consensus.cc:729] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688fcbfd26c442e086cecbe0b7234a95, State: Initialized, Role: FOLLOWER
I20260808 02:57:12.523229 4367 consensus_queue.cc:260] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.523590 4368 raft_consensus.cc:348] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.523844 4368 raft_consensus.cc:374] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:12.523865 4367 ts_tablet_manager.cc:1428] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 02:57:12.523972 4368 raft_consensus.cc:729] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e45cdac1d045c38dc2791340f7f922, State: Initialized, Role: FOLLOWER
I20260808 02:57:12.523993 4366 ts_tablet_manager.cc:1428] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:12.524235 4368 consensus_queue.cc:260] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.525193 4368 ts_tablet_manager.cc:1428] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:57:12.585892 4325 raft_consensus.cc:482] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:12.586148 4325 raft_consensus.cc:504] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.586864 4325 leader_election.cc:288] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851), a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729)
I20260808 02:57:12.587678 4141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0144ae57fd544c9fb70d3f1eb38af67e" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e45cdac1d045c38dc2791340f7f922" is_pre_election: true
I20260808 02:57:12.587695 3715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0144ae57fd544c9fb70d3f1eb38af67e" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52be984b29e4f99b1ff1d148b6fb368" is_pre_election: true
I20260808 02:57:12.587982 4141 raft_consensus.cc:2454] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 0.
I20260808 02:57:12.587991 3715 raft_consensus.cc:2454] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 0.
I20260808 02:57:12.588608 3815 leader_election.cc:302] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 688fcbfd26c442e086cecbe0b7234a95, a5e45cdac1d045c38dc2791340f7f922, c52be984b29e4f99b1ff1d148b6fb368; no voters:
I20260808 02:57:12.588953 4325 raft_consensus.cc:2781] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:12.589205 4325 raft_consensus.cc:482] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:12.589329 4325 raft_consensus.cc:3037] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:12.591957 4325 raft_consensus.cc:504] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.592691 4325 leader_election.cc:288] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 election: Requested vote from peers c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851), a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729)
I20260808 02:57:12.593168 3715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0144ae57fd544c9fb70d3f1eb38af67e" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52be984b29e4f99b1ff1d148b6fb368"
I20260808 02:57:12.593206 4141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0144ae57fd544c9fb70d3f1eb38af67e" candidate_uuid: "688fcbfd26c442e086cecbe0b7234a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e45cdac1d045c38dc2791340f7f922"
I20260808 02:57:12.593446 4141 raft_consensus.cc:3037] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:12.593457 3715 raft_consensus.cc:3037] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:12.596626 3715 raft_consensus.cc:2454] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 1.
I20260808 02:57:12.596655 4141 raft_consensus.cc:2454] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 688fcbfd26c442e086cecbe0b7234a95 in term 1.
I20260808 02:57:12.597215 3812 leader_election.cc:302] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688fcbfd26c442e086cecbe0b7234a95, a5e45cdac1d045c38dc2791340f7f922; no voters:
I20260808 02:57:12.597579 4325 raft_consensus.cc:2781] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:12.597805 4325 raft_consensus.cc:686] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 1 LEADER]: Becoming Leader. State: Replica: 688fcbfd26c442e086cecbe0b7234a95, State: Running, Role: LEADER
I20260808 02:57:12.598134 4325 consensus_queue.cc:237] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.601158 3476 catalog_manager.cc:5495] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130). New cstate: current_term: 1 leader_uuid: "688fcbfd26c442e086cecbe0b7234a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } health_report { overall_health: UNKNOWN } } }
I20260808 02:57:12.630957 4376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.624692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53476 (local address 127.0.205.130:40139)
0808 02:57:12.625527 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:12.625531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:12.625552 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:57:12.625608 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:12.625612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:12.625675 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:12.625764 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:12.626304 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.627404 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.628996 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.629438 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.629589 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.630133 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:12.630253 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:12.630361 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:12.630516 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 02:57:12.630599 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":461,"threads_started":1}
I20260808 02:57:12.640368 4141 raft_consensus.cc:1261] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Refusing update from remote peer 688fcbfd26c442e086cecbe0b7234a95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:12.640522 3715 raft_consensus.cc:1261] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Refusing update from remote peer 688fcbfd26c442e086cecbe0b7234a95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:12.641039 4325 consensus_queue.cc:1035] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:12.641393 4251 consensus_queue.cc:1035] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:12.686583 3474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:12.686882 3474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:12.689817 3474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31: Sending DeleteTablet for 3 replicas of tablet 0144ae57fd544c9fb70d3f1eb38af67e
I20260808 02:57:12.690779 3683 tablet_service.cc:1514] Processing DeleteTablet for tablet 0144ae57fd544c9fb70d3f1eb38af67e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:12 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:52448
I20260808 02:57:12.691102 4354 tablet_replica.cc:330] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368: stopping tablet replica
I20260808 02:57:12.691051 3885 tablet_service.cc:1514] Processing DeleteTablet for tablet 0144ae57fd544c9fb70d3f1eb38af67e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:12 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:35234
I20260808 02:57:12.691056 4110 tablet_service.cc:1514] Processing DeleteTablet for tablet 0144ae57fd544c9fb70d3f1eb38af67e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:12 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:59840
I20260808 02:57:12.691308 4354 raft_consensus.cc:2229] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:12.691370 4355 tablet_replica.cc:330] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922: stopping tablet replica
I20260808 02:57:12.691493 4354 raft_consensus.cc:2258] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:12.691627 4355 raft_consensus.cc:2229] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:12.691782 4356 tablet_replica.cc:330] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95: stopping tablet replica
I20260808 02:57:12.691844 4355 raft_consensus.cc:2258] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:12.692018 4356 raft_consensus.cc:2229] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:57:12.692348 4356 raft_consensus.cc:2258] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:12.692310 3474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:57:12.693011 3474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:12.693493 4355 ts_tablet_manager.cc:1905] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:12.693589 4354 ts_tablet_manager.cc:1905] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:12.693706 4356 ts_tablet_manager.cc:1905] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:12.700549 4355 ts_tablet_manager.cc:1918] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:57:12.700735 4355 log.cc:1196] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/0144ae57fd544c9fb70d3f1eb38af67e
I20260808 02:57:12.701248 3683 tablet_service.cc:1467] Processing CreateTablet for tablet 5d89ef5ad32d422fadab85c6c35b9f6b (DEFAULT_TABLE table=test-table [id=e61791a1c0dd444cb6797770c7ad87f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:12.701265 4110 tablet_service.cc:1467] Processing CreateTablet for tablet 5d89ef5ad32d422fadab85c6c35b9f6b (DEFAULT_TABLE table=test-table [id=e61791a1c0dd444cb6797770c7ad87f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:12.701692 3683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d89ef5ad32d422fadab85c6c35b9f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:12.701691 4110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d89ef5ad32d422fadab85c6c35b9f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:12.702104 3885 tablet_service.cc:1467] Processing CreateTablet for tablet 5d89ef5ad32d422fadab85c6c35b9f6b (DEFAULT_TABLE table=test-table [id=e61791a1c0dd444cb6797770c7ad87f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:12.702545 3885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d89ef5ad32d422fadab85c6c35b9f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:12.703850 4356 ts_tablet_manager.cc:1918] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:57:12.704007 4356 log.cc:1196] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/0144ae57fd544c9fb70d3f1eb38af67e
I20260808 02:57:12.704541 4356 ts_tablet_manager.cc:1939] T 0144ae57fd544c9fb70d3f1eb38af67e P 688fcbfd26c442e086cecbe0b7234a95: Deleting consensus metadata
I20260808 02:57:12.705592 3454 catalog_manager.cc:4841] TS 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130:40139): tablet 0144ae57fd544c9fb70d3f1eb38af67e (table test-table [id=6ae70dcdef3b4ef38d79cfef012db8e0]) successfully deleted
I20260808 02:57:12.708297 4366 tablet_bootstrap.cc:492] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368: Bootstrap starting.
I20260808 02:57:12.708293 4368 tablet_bootstrap.cc:492] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922: Bootstrap starting.
I20260808 02:57:12.700853 4354 ts_tablet_manager.cc:1918] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:57:12.708721 4354 log.cc:1196] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/0144ae57fd544c9fb70d3f1eb38af67e
I20260808 02:57:12.709281 4354 ts_tablet_manager.cc:1939] T 0144ae57fd544c9fb70d3f1eb38af67e P c52be984b29e4f99b1ff1d148b6fb368: Deleting consensus metadata
I20260808 02:57:12.709991 4366 tablet_bootstrap.cc:654] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:12.710139 4368 tablet_bootstrap.cc:654] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:12.710451 3456 catalog_manager.cc:4841] TS c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851): tablet 0144ae57fd544c9fb70d3f1eb38af67e (table test-table [id=6ae70dcdef3b4ef38d79cfef012db8e0]) successfully deleted
I20260808 02:57:12.711102 4367 tablet_bootstrap.cc:492] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95: Bootstrap starting.
I20260808 02:57:12.701256 4355 ts_tablet_manager.cc:1939] T 0144ae57fd544c9fb70d3f1eb38af67e P a5e45cdac1d045c38dc2791340f7f922: Deleting consensus metadata
I20260808 02:57:12.712527 4368 tablet_bootstrap.cc:492] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922: No bootstrap required, opened a new log
I20260808 02:57:12.712707 4368 ts_tablet_manager.cc:1397] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 02:57:12.712765 4367 tablet_bootstrap.cc:654] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:12.713557 4368 raft_consensus.cc:348] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.713852 4368 raft_consensus.cc:374] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:12.713976 4368 raft_consensus.cc:729] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e45cdac1d045c38dc2791340f7f922, State: Initialized, Role: FOLLOWER
I20260808 02:57:12.714069 3454 catalog_manager.cc:4841] TS a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729): tablet 0144ae57fd544c9fb70d3f1eb38af67e (table test-table [id=6ae70dcdef3b4ef38d79cfef012db8e0]) successfully deleted
I20260808 02:57:12.714238 4368 consensus_queue.cc:260] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.714871 4368 ts_tablet_manager.cc:1428] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:57:12.715503 4366 tablet_bootstrap.cc:492] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368: No bootstrap required, opened a new log
I20260808 02:57:12.715677 4366 ts_tablet_manager.cc:1397] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260808 02:57:12.716285 4367 tablet_bootstrap.cc:492] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95: No bootstrap required, opened a new log
I20260808 02:57:12.716300 4366 raft_consensus.cc:348] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.716465 4367 ts_tablet_manager.cc:1397] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260808 02:57:12.716598 4366 raft_consensus.cc:374] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:12.716701 4366 raft_consensus.cc:729] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52be984b29e4f99b1ff1d148b6fb368, State: Initialized, Role: FOLLOWER
I20260808 02:57:12.716966 4366 consensus_queue.cc:260] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.717270 4367 raft_consensus.cc:348] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.717494 4366 ts_tablet_manager.cc:1428] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 02:57:12.717532 4367 raft_consensus.cc:374] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:12.717617 4367 raft_consensus.cc:729] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688fcbfd26c442e086cecbe0b7234a95, State: Initialized, Role: FOLLOWER
I20260808 02:57:12.717855 4367 consensus_queue.cc:260] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.718451 4367 ts_tablet_manager.cc:1428] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:57:12.736631 4373 raft_consensus.cc:482] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:12.736878 4373 raft_consensus.cc:504] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.738700 4373 leader_election.cc:288] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851), 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130:40139)
I20260808 02:57:12.748375 4376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.739331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:48309 (local address 127.0.205.130:40139)
0808 02:57:12.739591 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:12.739598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:12.740766 (+ 1168us) server_negotiation.cc:407] Connection header received
0808 02:57:12.741061 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:12.741066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:12.741142 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:12.741228 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:12.742120 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.742996 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.745996 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.747234 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.747433 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.747512 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:12.748001 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 02:57:12.748138 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:57:12.748492 4394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.738803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:33851 (local address 127.0.205.131:36981)
0808 02:57:12.740411 (+ 1608us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:12.740435 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:12.740663 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:12.741310 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:12.741314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:12.741344 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:12.741720 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:12.741731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.743024 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.743030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:12.745784 (+ 2754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:12.745800 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.747536 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.747543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:12.747741 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.747831 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 02:57:12.747933 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 02:57:12.748068 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1453,"mutex_wait_us":170,"spinlock_wait_cycles":2944,"thread_start_us":1593,"threads_started":1}
I20260808 02:57:12.748509 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.739041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:36981 (local address 127.0.205.129:33851)
0808 02:57:12.739928 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:12.739936 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:12.740528 (+ 592us) server_negotiation.cc:407] Connection header received
0808 02:57:12.740851 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:12.740858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:12.740936 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:12.741053 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:12.742014 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.742861 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.746002 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.747234 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.747451 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.747519 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:12.747969 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 02:57:12.748142 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":531,"threads_started":1}
I20260808 02:57:12.749174 3715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d89ef5ad32d422fadab85c6c35b9f6b" candidate_uuid: "a5e45cdac1d045c38dc2791340f7f922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52be984b29e4f99b1ff1d148b6fb368" is_pre_election: true
I20260808 02:57:12.749462 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.738960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:40139 (local address 127.0.205.131:48309)
0808 02:57:12.740665 (+ 1705us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:12.740682 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:12.740878 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:12.741319 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:12.741326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:12.741426 (+ 100us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:12.741947 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:12.741962 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.743126 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.743133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:12.745784 (+ 2651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:12.745800 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.747508 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.747521 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:12.747745 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.747828 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 02:57:12.748876 (+ 1048us) client_negotiation.cc:240] Negotiation successful
0808 02:57:12.749024 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1586,"mutex_wait_us":227,"spinlock_wait_cycles":2688,"thread_start_us":923,"threads_started":1}
I20260808 02:57:12.749464 3715 raft_consensus.cc:2454] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5e45cdac1d045c38dc2791340f7f922 in term 0.
I20260808 02:57:12.750079 4030 leader_election.cc:302] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5e45cdac1d045c38dc2791340f7f922, c52be984b29e4f99b1ff1d148b6fb368; no voters:
I20260808 02:57:12.750205 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d89ef5ad32d422fadab85c6c35b9f6b" candidate_uuid: "a5e45cdac1d045c38dc2791340f7f922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688fcbfd26c442e086cecbe0b7234a95" is_pre_election: true
I20260808 02:57:12.750510 4373 raft_consensus.cc:2781] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:12.750517 3922 raft_consensus.cc:2454] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5e45cdac1d045c38dc2791340f7f922 in term 0.
I20260808 02:57:12.750659 4373 raft_consensus.cc:482] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:12.750802 4373 raft_consensus.cc:3037] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:12.753085 4373 raft_consensus.cc:504] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.753703 4373 leader_election.cc:288] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [CANDIDATE]: Term 1 election: Requested vote from peers c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851), 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130:40139)
I20260808 02:57:12.754168 3715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d89ef5ad32d422fadab85c6c35b9f6b" candidate_uuid: "a5e45cdac1d045c38dc2791340f7f922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52be984b29e4f99b1ff1d148b6fb368"
I20260808 02:57:12.754249 3922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d89ef5ad32d422fadab85c6c35b9f6b" candidate_uuid: "a5e45cdac1d045c38dc2791340f7f922" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688fcbfd26c442e086cecbe0b7234a95"
I20260808 02:57:12.754369 3715 raft_consensus.cc:3037] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:12.754485 3922 raft_consensus.cc:3037] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:12.756811 3922 raft_consensus.cc:2454] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5e45cdac1d045c38dc2791340f7f922 in term 1.
I20260808 02:57:12.756836 3715 raft_consensus.cc:2454] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5e45cdac1d045c38dc2791340f7f922 in term 1.
I20260808 02:57:12.757283 4027 leader_election.cc:302] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688fcbfd26c442e086cecbe0b7234a95, a5e45cdac1d045c38dc2791340f7f922; no voters:
I20260808 02:57:12.757614 4373 raft_consensus.cc:2781] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:12.758165 4373 raft_consensus.cc:686] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 1 LEADER]: Becoming Leader. State: Replica: a5e45cdac1d045c38dc2791340f7f922, State: Running, Role: LEADER
I20260808 02:57:12.758597 4373 consensus_queue.cc:237] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } }
I20260808 02:57:12.762467 3475 catalog_manager.cc:5495] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 reported cstate change: term changed from 0 to 1, leader changed from <none> to a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131). New cstate: current_term: 1 leader_uuid: "a5e45cdac1d045c38dc2791340f7f922" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5e45cdac1d045c38dc2791340f7f922" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43729 } health_report { overall_health: HEALTHY } } }
I20260808 02:57:12.786195 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.780075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.0.205.131:43729)
0808 02:57:12.780729 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:12.780733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:12.780753 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:57:12.780835 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:12.780838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:12.780923 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:12.781064 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:12.781435 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.782325 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.783785 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.784301 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.784407 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.784859 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:12.785568 (+ 709us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:12.785678 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:12.785770 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 02:57:12.785833 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":398,"threads_started":1}
I20260808 02:57:12.802218 4376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.796390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:60409 (local address 127.0.205.130:40139)
0808 02:57:12.796564 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:12.796592 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:12.797112 (+ 520us) server_negotiation.cc:407] Connection header received
0808 02:57:12.797333 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:12.797337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:12.797422 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:12.797523 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:12.798082 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.798924 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.800358 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.800792 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.800962 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.801407 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:12.801516 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:12.801617 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:12.801758 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 02:57:12.801849 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 02:57:12.802304 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.796379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:40139 (local address 127.0.205.131:60409)
0808 02:57:12.797008 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:12.797025 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:12.797191 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:12.797538 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:12.797542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:12.797554 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:57:12.797938 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:12.797948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.799070 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.799076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:12.800216 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:12.800228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.801020 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.801029 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:12.801168 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.801222 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:57:12.801620 (+ 398us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:57:12.801634 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 02:57:12.801733 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 02:57:12.801875 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":321,"threads_started":1}
I20260808 02:57:12.810576 3922 raft_consensus.cc:1261] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Refusing update from remote peer a5e45cdac1d045c38dc2791340f7f922: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:12.811196 3715 raft_consensus.cc:1261] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Refusing update from remote peer a5e45cdac1d045c38dc2791340f7f922: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:12.811232 4398 consensus_queue.cc:1035] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688fcbfd26c442e086cecbe0b7234a95" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:12.811805 4373 consensus_queue.cc:1035] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c52be984b29e4f99b1ff1d148b6fb368" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:12.815976 4326 mvcc.cc:205] Tried to move back new op lower bound from 7316102483185897472 to 7316102482983596032. Current Snapshot: MvccSnapshot[applied={T|T < 7316102483185897472}]
I20260808 02:57:12.840618 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.834770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:51667 (local address 127.0.205.131:43729)
0808 02:57:12.834954 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:12.834961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:12.835199 (+ 238us) server_negotiation.cc:407] Connection header received
0808 02:57:12.835415 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:12.835419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:12.835479 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:12.835549 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:12.836052 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.837155 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.838905 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.839315 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.839468 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.839865 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:12.839960 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:12.840061 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:12.840228 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 02:57:12.840303 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:57:12.840684 4415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:12.834460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43729 (local address 127.0.205.130:51667)
0808 02:57:12.835111 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:12.835127 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:12.835280 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:12.835556 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:12.835559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:12.835568 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:57:12.835860 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:12.835869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.837302 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.837308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:12.838751 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:12.838765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:12.839503 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:12.839509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:12.839659 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:12.839697 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:57:12.840092 (+ 395us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:57:12.840103 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 02:57:12.840203 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 02:57:12.840340 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":387,"threads_started":1}
W20260808 02:57:12.842195 4326 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 02:57:14.463603 3476 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:14.463856 3476 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:14.468268 3476 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27f9e97edeb748c4bbfc4077dcd59d31: Sending DeleteTablet for 3 replicas of tablet 5d89ef5ad32d422fadab85c6c35b9f6b
I20260808 02:57:14.469420 3683 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d89ef5ad32d422fadab85c6c35b9f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:14 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:52448
I20260808 02:57:14.469412 3885 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d89ef5ad32d422fadab85c6c35b9f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:14 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:35234
I20260808 02:57:14.469718 4110 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d89ef5ad32d422fadab85c6c35b9f6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:14 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:59840
I20260808 02:57:14.470094 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 3584
I20260808 02:57:14.482345 4443 tablet_replica.cc:330] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95: stopping tablet replica
I20260808 02:57:14.482326 4445 tablet_replica.cc:330] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368: stopping tablet replica
I20260808 02:57:14.482767 4443 raft_consensus.cc:2229] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:14.482364 4447 tablet_replica.cc:330] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922: stopping tablet replica
I20260808 02:57:14.483012 4445 raft_consensus.cc:2229] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:14.483091 4443 raft_consensus.cc:2258] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:14.483278 4445 raft_consensus.cc:2258] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:14.483215 4447 raft_consensus.cc:2229] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:57:14.484362 4443 ts_tablet_manager.cc:1905] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:14.484459 4445 ts_tablet_manager.cc:1905] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:14.484694 4447 raft_consensus.cc:2258] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:14.485577 4447 ts_tablet_manager.cc:1905] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:14.491612 4445 ts_tablet_manager.cc:1918] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:57:14.491647 4443 ts_tablet_manager.cc:1918] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:57:14.491756 4445 log.cc:1196] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/5d89ef5ad32d422fadab85c6c35b9f6b
I20260808 02:57:14.491977 4443 log.cc:1196] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/5d89ef5ad32d422fadab85c6c35b9f6b
I20260808 02:57:14.492201 4445 ts_tablet_manager.cc:1939] T 5d89ef5ad32d422fadab85c6c35b9f6b P c52be984b29e4f99b1ff1d148b6fb368: Deleting consensus metadata
I20260808 02:57:14.492628 4443 ts_tablet_manager.cc:1939] T 5d89ef5ad32d422fadab85c6c35b9f6b P 688fcbfd26c442e086cecbe0b7234a95: Deleting consensus metadata
I20260808 02:57:14.492739 4447 ts_tablet_manager.cc:1918] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:57:14.492879 4447 log.cc:1196] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/5d89ef5ad32d422fadab85c6c35b9f6b
I20260808 02:57:14.493378 4447 ts_tablet_manager.cc:1939] T 5d89ef5ad32d422fadab85c6c35b9f6b P a5e45cdac1d045c38dc2791340f7f922: Deleting consensus metadata
I20260808 02:57:14.493476 3456 catalog_manager.cc:4841] TS c52be984b29e4f99b1ff1d148b6fb368 (127.0.205.129:33851): tablet 5d89ef5ad32d422fadab85c6c35b9f6b (table test-table [id=e61791a1c0dd444cb6797770c7ad87f3]) successfully deleted
I20260808 02:57:14.494174 3454 catalog_manager.cc:4841] TS 688fcbfd26c442e086cecbe0b7234a95 (127.0.205.130:40139): tablet 5d89ef5ad32d422fadab85c6c35b9f6b (table test-table [id=e61791a1c0dd444cb6797770c7ad87f3]) successfully deleted
I20260808 02:57:14.494571 3454 catalog_manager.cc:4841] TS a5e45cdac1d045c38dc2791340f7f922 (127.0.205.131:43729): tablet 5d89ef5ad32d422fadab85c6c35b9f6b (table test-table [id=e61791a1c0dd444cb6797770c7ad87f3]) successfully deleted
Aug 08 02:57:14 dist-test-slave-6ntk krb5kdc[3418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:14.501307 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:14.472915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.0.205.129:33851)
0808 02:57:14.482819 (+ 9904us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:14.482826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:14.483941 (+ 1115us) server_negotiation.cc:407] Connection header received
0808 02:57:14.484010 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:14.484014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:14.484092 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:14.484197 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:14.486501 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:14.487658 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:14.489543 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:14.490235 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:14.490363 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:14.495806 (+ 5443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:14.495822 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:14.495839 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:14.495876 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:14.498828 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:14.499507 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:14.499514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:14.499516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:14.499652 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:14.499989 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:14.499992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:14.499994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:14.500357 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:14.500583 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:14.500839 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 02:57:14.501039 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9753,"thread_start_us":9641,"threads_started":1}
W20260808 02:57:14.502091 3771 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46568
W20260808 02:57:14.502745 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 3584: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:57:14.503018 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 3584
I20260808 02:57:14.537045 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 3790
Aug 08 02:57:14 dist-test-slave-6ntk krb5kdc[3418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:57:14.553135 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:14.537788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.0.205.130:40139)
0808 02:57:14.538424 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:14.538428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:14.538445 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:57:14.538502 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:14.538506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:14.538572 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:14.538665 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:14.540027 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:14.540889 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:14.542348 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:14.542752 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:14.542859 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:14.547297 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:14.547314 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:14.547331 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:14.547377 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:14.550465 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:14.551168 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:14.551176 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:14.551178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:14.551321 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:14.551659 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:14.551662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:14.551664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:14.552043 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:14.552268 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:14.552523 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 02:57:14.552708 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":376,"threads_started":1}
W20260808 02:57:14.554066 3991 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53482
W20260808 02:57:14.554762 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 3790: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:57:14.554925 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 3790
I20260808 02:57:14.591864 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 4005
Aug 08 02:57:14 dist-test-slave-6ntk krb5kdc[3418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:57:14.608026 4453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:14.592662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.0.205.131:43729)
0808 02:57:14.593200 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:14.593205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:14.593222 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:57:14.593279 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:14.593283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:14.593368 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:14.593463 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:14.594880 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:14.595621 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:14.596823 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:14.597265 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:14.597361 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:14.601831 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:14.601918 (+ 87us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:14.601945 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:14.601984 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:14.604823 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:14.605478 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:14.605484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:14.605487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:14.605613 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:14.606118 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:14.606122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:14.606124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:14.606497 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:14.606755 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:14.607160 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 02:57:14.607518 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":303,"threads_started":1}
W20260808 02:57:14.608934 4202 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43834
W20260808 02:57:14.609591 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 4005: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:57:14.609746 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 4005
I20260808 02:57:14.645709 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 3434
I20260808 02:57:14.647094 3540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:14.768432 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 3434
2026-08-08T02:57:14Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8560 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:57:14 dist-test-slave-6ntk krb5kdc[4461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:57:14 dist-test-slave-6ntk krb5kdc[4461](info): set up 2 sockets
Aug 08 02:57:14 dist-test-slave-6ntk krb5kdc[4461](info): commencing operation
krb5kdc: starting...
W20260808 02:57:16.068701 822 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:57:16 dist-test-slave-6ntk krb5kdc[4461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:57:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:57:16.299592 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:39553
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:16.634943 4479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:16.635352 4479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:16.635527 4479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:16.642871 4479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:16.643046 4479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:16.643162 4479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:16.643281 4479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:16.643371 4479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:16.653448 4479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:39553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:39553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:16.654942 4479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:16.656481 4479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:16.669114 4487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:16.669157 4489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:16.669363 4479 server_base.cc:1034] running on GCE node
W20260808 02:57:16.669562 4486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:16.670751 4479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:16.672251 4479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:16.673511 4479 hybrid_clock.cc:648] HybridClock initialized: now 1786157836673458 us; error 45 us; skew 500 ppm
I20260808 02:57:16.673923 4479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:16 dist-test-slave-6ntk krb5kdc[4461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157836, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:16.677161 4479 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:57:16.678676 4479 webserver.cc:460] Webserver started at http://127.0.205.190:38237/ using document root <none> and password file <none>
I20260808 02:57:16.679361 4479 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:16.679462 4479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:16.679723 4479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:16.682726 4479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "b69eccdd8d024c0bad4d7a2d13465fa9"
format_stamp: "Formatted at 2026-08-08 02:57:16 on dist-test-slave-6ntk"
server_key: "073b6d053a390ffca641fe9148531ee1"
server_key_iv: "24a99c2d4109a026a303d4a49f6d80a1"
server_key_version: "encryptionkey@0"
I20260808 02:57:16.683471 4479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "b69eccdd8d024c0bad4d7a2d13465fa9"
format_stamp: "Formatted at 2026-08-08 02:57:16 on dist-test-slave-6ntk"
server_key: "073b6d053a390ffca641fe9148531ee1"
server_key_iv: "24a99c2d4109a026a303d4a49f6d80a1"
server_key_version: "encryptionkey@0"
I20260808 02:57:16.688732 4479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 02:57:16.692240 4499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:16.693614 4479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:57:16.693768 4479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "b69eccdd8d024c0bad4d7a2d13465fa9"
format_stamp: "Formatted at 2026-08-08 02:57:16 on dist-test-slave-6ntk"
server_key: "073b6d053a390ffca641fe9148531ee1"
server_key_iv: "24a99c2d4109a026a303d4a49f6d80a1"
server_key_version: "encryptionkey@0"
I20260808 02:57:16.694020 4479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:16.710853 4479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:16.714895 4479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:16.715140 4479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:16.734779 4479 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:39553
I20260808 02:57:16.734800 4578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:39553 every 8 connection(s)
I20260808 02:57:16.736225 4479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:16.740120 4580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:16.741739 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 4479
I20260808 02:57:16.742038 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:16.742904 822 external_mini_cluster.cc:1428] Setting key 2d11472f101325d68c6bd4bb627934cb
I20260808 02:57:16.755744 4580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9: Bootstrap starting.
Aug 08 02:57:16 dist-test-slave-6ntk krb5kdc[4461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:16.759652 4580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:16.760807 4580 log.cc:824] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:16.763846 4580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9: No bootstrap required, opened a new log
I20260808 02:57:16.767315 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:16.745199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49140 (local address 127.0.205.190:39553)
0808 02:57:16.746161 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:16.746174 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:16.746224 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:57:16.747897 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:16.747921 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:16.748344 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:16.748687 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:16.750645 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:16.751968 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:16.753811 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:16.754580 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:16.754707 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:16.759524 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:16.759532 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:16.759550 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:16.759593 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:16.763028 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:16.763741 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:16.763749 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:16.763755 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:16.763990 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:16.764398 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:16.764404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:16.764407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:16.764954 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:16.765251 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:16.765814 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 02:57:16.766282 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":454,"threads_started":1}
I20260808 02:57:16.772135 4580 raft_consensus.cc:348] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69eccdd8d024c0bad4d7a2d13465fa9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39553 } }
I20260808 02:57:16.772517 4580 raft_consensus.cc:374] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:16.772655 4580 raft_consensus.cc:729] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b69eccdd8d024c0bad4d7a2d13465fa9, State: Initialized, Role: FOLLOWER
I20260808 02:57:16.773303 4580 consensus_queue.cc:260] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69eccdd8d024c0bad4d7a2d13465fa9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39553 } }
I20260808 02:57:16.773561 4580 raft_consensus.cc:388] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:16.773696 4580 raft_consensus.cc:482] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:16.773936 4580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:16.776131 4580 raft_consensus.cc:504] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69eccdd8d024c0bad4d7a2d13465fa9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39553 } }
I20260808 02:57:16.776662 4580 leader_election.cc:302] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b69eccdd8d024c0bad4d7a2d13465fa9; no voters:
I20260808 02:57:16.777271 4580 leader_election.cc:288] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:16.777441 4585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:16.778172 4585 raft_consensus.cc:686] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [term 1 LEADER]: Becoming Leader. State: Replica: b69eccdd8d024c0bad4d7a2d13465fa9, State: Running, Role: LEADER
I20260808 02:57:16.778724 4585 consensus_queue.cc:237] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69eccdd8d024c0bad4d7a2d13465fa9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39553 } }
I20260808 02:57:16.779515 4580 sys_catalog.cc:564] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:16.781596 4588 sys_catalog.cc:455] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b69eccdd8d024c0bad4d7a2d13465fa9. Latest consensus state: current_term: 1 leader_uuid: "b69eccdd8d024c0bad4d7a2d13465fa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69eccdd8d024c0bad4d7a2d13465fa9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39553 } } }
I20260808 02:57:16.781535 4587 sys_catalog.cc:455] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b69eccdd8d024c0bad4d7a2d13465fa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b69eccdd8d024c0bad4d7a2d13465fa9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39553 } } }
I20260808 02:57:16.782019 4587 sys_catalog.cc:458] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:16.782019 4588 sys_catalog.cc:458] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:16.783047 4593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:16.789005 4593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:16.799227 4593 catalog_manager.cc:1347] Generated new cluster ID: f51d54b5d16d4764860e983cdb6472bf
I20260808 02:57:16.799360 4593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:16.826498 4593 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:16.828189 4593 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:16.847218 4593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b69eccdd8d024c0bad4d7a2d13465fa9: Generated new TSK 0
I20260808 02:57:16.848245 4593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:57:16.908618 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39553
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:17.257530 4615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:17.257834 4615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:17.258054 4615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:17.265848 4615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:17.266304 4615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:17.266531 4615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:17.277257 4615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:17.278738 4615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:17.280336 4615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:17.294448 4622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:17.294513 4623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:17.294544 4628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:17.296244 4615 server_base.cc:1034] running on GCE node
I20260808 02:57:17.296912 4615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:17.298075 4615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:17.299283 4615 hybrid_clock.cc:648] HybridClock initialized: now 1786157837299265 us; error 100 us; skew 500 ppm
I20260808 02:57:17.299574 4615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:17 dist-test-slave-6ntk krb5kdc[4461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:17.302781 4615 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:57:17.304308 4615 webserver.cc:460] Webserver started at http://127.0.205.129:42581/ using document root <none> and password file <none>
I20260808 02:57:17.305043 4615 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:17.305181 4615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:17.305519 4615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:17.308537 4615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "856a04a842b940c7892f8b3bfd4f942a"
format_stamp: "Formatted at 2026-08-08 02:57:17 on dist-test-slave-6ntk"
server_key: "f7a1b739e6d16d232fae1e0add420ec1"
server_key_iv: "954a4287375a8a3aa74b3a25a0520b7b"
server_key_version: "encryptionkey@0"
I20260808 02:57:17.309347 4615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "856a04a842b940c7892f8b3bfd4f942a"
format_stamp: "Formatted at 2026-08-08 02:57:17 on dist-test-slave-6ntk"
server_key: "f7a1b739e6d16d232fae1e0add420ec1"
server_key_iv: "954a4287375a8a3aa74b3a25a0520b7b"
server_key_version: "encryptionkey@0"
I20260808 02:57:17.314721 4615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 02:57:17.318256 4637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:17.319712 4615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 02:57:17.319911 4615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "856a04a842b940c7892f8b3bfd4f942a"
format_stamp: "Formatted at 2026-08-08 02:57:17 on dist-test-slave-6ntk"
server_key: "f7a1b739e6d16d232fae1e0add420ec1"
server_key_iv: "954a4287375a8a3aa74b3a25a0520b7b"
server_key_version: "encryptionkey@0"
I20260808 02:57:17.320137 4615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:17.358534 4615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:17.362993 4615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:17.363355 4615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:17.364485 4615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:17.366039 4615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:17.366154 4615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:17.366321 4615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:17.366408 4615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:17.401763 4615 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:37149
I20260808 02:57:17.401858 4802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:37149 every 8 connection(s)
I20260808 02:57:17.403199 4615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:17.412303 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 4615
I20260808 02:57:17.412549 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:57:17.413177 822 external_mini_cluster.cc:1428] Setting key dd8b9d13ccfb470905843420f76824eb
Aug 08 02:57:17 dist-test-slave-6ntk krb5kdc[4461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:17.424623 4805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:17.405857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:60783 (local address 127.0.205.190:39553)
0808 02:57:17.406425 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:17.406430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:17.407336 (+ 906us) server_negotiation.cc:407] Connection header received
0808 02:57:17.409221 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:17.409227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:17.409324 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:17.409451 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:17.411080 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:17.411838 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:17.413568 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:17.414160 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:17.414305 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:17.418588 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:17.418593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:17.418596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:17.418629 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:17.421302 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:17.422170 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:17.422174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:17.422177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:17.422305 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:17.422769 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:17.422780 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:17.422783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:17.423051 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:17.423218 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:17.424105 (+ 887us) server_negotiation.cc:299] Negotiation successful
0808 02:57:17.424287 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":271,"threads_started":1}
I20260808 02:57:17.426261 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:17.406326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:39553 (local address 127.0.205.129:60783)
0808 02:57:17.407085 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:17.407132 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:17.408937 (+ 1805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:17.409690 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:17.409701 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:17.410308 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:17.410886 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:17.410905 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:17.412012 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:17.412016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:17.413425 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:17.413438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:17.414441 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:17.414449 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:17.414746 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:17.415776 (+ 1030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:17.415799 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:17.418356 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:17.421462 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:17.421472 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:17.421489 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:17.422031 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:17.422422 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:17.422427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:17.422430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:17.422639 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:17.423184 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:17.423193 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:17.423618 (+ 425us) client_negotiation.cc:769] Sending connection context
0808 02:57:17.423944 (+ 326us) client_negotiation.cc:240] Negotiation successful
0808 02:57:17.424556 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":310,"threads_started":1}
I20260808 02:57:17.427983 4803 heartbeater.cc:347] Connected to a master server at 127.0.205.190:39553
I20260808 02:57:17.428316 4803 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:17.429100 4803 heartbeater.cc:511] Master 127.0.205.190:39553 requested a full tablet report, sending...
I20260808 02:57:17.431432 4526 ts_manager.cc:194] Registered new tserver with Master: 856a04a842b940c7892f8b3bfd4f942a (127.0.205.129:37149)
I20260808 02:57:17.433036 4526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:60783
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:57:17.466104 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39553
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:17.814260 4810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:17.814613 4810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:17.814798 4810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:17.822305 4810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:17.822465 4810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:17.822650 4810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:17.833030 4810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:17.834590 4810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:17.836030 4810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:17.850376 4822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:17.850382 4818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:17.850425 4817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:17.851428 4810 server_base.cc:1034] running on GCE node
I20260808 02:57:17.852156 4810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:17.853158 4810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:17.854413 4810 hybrid_clock.cc:648] HybridClock initialized: now 1786157837854340 us; error 46 us; skew 500 ppm
I20260808 02:57:17.854751 4810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:17 dist-test-slave-6ntk krb5kdc[4461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:17.863655 4810 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:57:17.865326 4810 webserver.cc:460] Webserver started at http://127.0.205.130:43085/ using document root <none> and password file <none>
I20260808 02:57:17.866184 4810 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:17.866291 4810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:17.866575 4810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:17.869500 4810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "6c95fca8072f42e889140ae8f0d1f5ff"
format_stamp: "Formatted at 2026-08-08 02:57:17 on dist-test-slave-6ntk"
server_key: "b98cc10bd6bb3cb1f0ed81aabcec15f0"
server_key_iv: "e20ae97565797c93ba8b2c3eb13008e4"
server_key_version: "encryptionkey@0"
I20260808 02:57:17.870321 4810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "6c95fca8072f42e889140ae8f0d1f5ff"
format_stamp: "Formatted at 2026-08-08 02:57:17 on dist-test-slave-6ntk"
server_key: "b98cc10bd6bb3cb1f0ed81aabcec15f0"
server_key_iv: "e20ae97565797c93ba8b2c3eb13008e4"
server_key_version: "encryptionkey@0"
I20260808 02:57:17.875521 4810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:57:17.879034 4830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:17.880565 4810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:57:17.880697 4810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "6c95fca8072f42e889140ae8f0d1f5ff"
format_stamp: "Formatted at 2026-08-08 02:57:17 on dist-test-slave-6ntk"
server_key: "b98cc10bd6bb3cb1f0ed81aabcec15f0"
server_key_iv: "e20ae97565797c93ba8b2c3eb13008e4"
server_key_version: "encryptionkey@0"
I20260808 02:57:17.880864 4810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:17.943573 4810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:17.948290 4810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:17.948560 4810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:17.949497 4810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:17.950964 4810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:17.951041 4810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:17.951131 4810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:17.951176 4810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:17.986454 4810 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:33421
I20260808 02:57:17.986474 4990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:33421 every 8 connection(s)
I20260808 02:57:17.987835 4810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:17.990211 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 4810
I20260808 02:57:17.990463 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:57:17.991045 822 external_mini_cluster.cc:1428] Setting key 93a6eb21fc91169bdac7ab8096c63fda
Aug 08 02:57:18 dist-test-slave-6ntk krb5kdc[4461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:18.011933 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:17.991924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:58081 (local address 127.0.205.190:39553)
0808 02:57:17.992414 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:17.992420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:17.992443 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:57:17.994415 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:17.994424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:17.994514 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:17.994640 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:17.996119 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:17.997265 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:17.999228 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:17.999988 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:18.000135 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:18.003856 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:18.003865 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:18.003873 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:18.003922 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:18.007932 (+ 4010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:18.008806 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:18.008814 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:18.008821 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:18.008988 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:18.009534 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:18.009542 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:18.009548 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:18.009991 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:18.010160 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:18.011124 (+ 964us) server_negotiation.cc:299] Negotiation successful
0808 02:57:18.011417 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":239,"threads_started":1}
I20260808 02:57:18.012676 4994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:17.991300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:39553 (local address 127.0.205.130:58081)
0808 02:57:17.992229 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:17.992280 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:17.994134 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:17.994838 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:17.994846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:17.995351 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:17.995937 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:17.995954 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:17.997442 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:17.997446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:17.999091 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:17.999103 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:18.000197 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:18.000201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:18.000414 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:18.001132 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:18.001154 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:18.003616 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:18.008101 (+ 4485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:18.008109 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:18.008123 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:18.008648 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:18.009108 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:18.009111 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:18.009114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:18.009389 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:18.010134 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:18.010141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:18.010682 (+ 541us) client_negotiation.cc:769] Sending connection context
0808 02:57:18.010953 (+ 271us) client_negotiation.cc:240] Negotiation successful
0808 02:57:18.011417 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":330,"threads_started":1}
I20260808 02:57:18.014453 4992 heartbeater.cc:347] Connected to a master server at 127.0.205.190:39553
I20260808 02:57:18.014814 4992 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
I20260808 02:57:18.015643 4992 heartbeater.cc:511] Master 127.0.205.190:39553 requested a full tablet report, sending...
I20260808 02:57:18.017444 4526 ts_manager.cc:194] Registered new tserver with Master: 6c95fca8072f42e889140ae8f0d1f5ff (127.0.205.130:33421)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
I20260808 02:57:18.018543 4526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:58081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:57:18.046088 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39553
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:18.403556 5006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:18.403848 5006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:18.404021 5006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:18.412180 5006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:18.412303 5006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:18.412498 5006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:18.422674 5006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:18.424139 5006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:18.425563 5006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:57:18.436230 4803 heartbeater.cc:503] Master 127.0.205.190:39553 was elected leader, sending a full tablet report...
W20260808 02:57:18.439992 5015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:18.440387 5017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:18.440410 5014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:18.440984 5006 server_base.cc:1034] running on GCE node
I20260808 02:57:18.441674 5006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:18.442835 5006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:18.444087 5006 hybrid_clock.cc:648] HybridClock initialized: now 1786157838444016 us; error 61 us; skew 500 ppm
I20260808 02:57:18.444428 5006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:18 dist-test-slave-6ntk krb5kdc[4461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:18.447877 5006 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:57:18.449517 5006 webserver.cc:460] Webserver started at http://127.0.205.131:46509/ using document root <none> and password file <none>
I20260808 02:57:18.450284 5006 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:18.450393 5006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:18.450670 5006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:18.453686 5006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "f29aafe6ebcd487191a5c35c29534dc6"
format_stamp: "Formatted at 2026-08-08 02:57:18 on dist-test-slave-6ntk"
server_key: "d08c121ea094717369c62829f13715f6"
server_key_iv: "01c69a56f400318c01d6123d5198fb2e"
server_key_version: "encryptionkey@0"
I20260808 02:57:18.454563 5006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "f29aafe6ebcd487191a5c35c29534dc6"
format_stamp: "Formatted at 2026-08-08 02:57:18 on dist-test-slave-6ntk"
server_key: "d08c121ea094717369c62829f13715f6"
server_key_iv: "01c69a56f400318c01d6123d5198fb2e"
server_key_version: "encryptionkey@0"
I20260808 02:57:18.460333 5006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 02:57:18.463943 5026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:18.465310 5006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:57:18.465515 5006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "f29aafe6ebcd487191a5c35c29534dc6"
format_stamp: "Formatted at 2026-08-08 02:57:18 on dist-test-slave-6ntk"
server_key: "d08c121ea094717369c62829f13715f6"
server_key_iv: "01c69a56f400318c01d6123d5198fb2e"
server_key_version: "encryptionkey@0"
I20260808 02:57:18.465746 5006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:18.483935 5006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:18.488866 5006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:18.489180 5006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:18.490345 5006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:18.491765 5006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:18.491842 5006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:18.491940 5006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:18.491984 5006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:18.527359 5006 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:40539
I20260808 02:57:18.527410 5186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:40539 every 8 connection(s)
I20260808 02:57:18.528755 5006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:57:18.529249 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 5006
I20260808 02:57:18.529474 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:57:18.530153 822 external_mini_cluster.cc:1428] Setting key faa638348abe5b5943ec0203db1d3fdc
Aug 08 02:57:18 dist-test-slave-6ntk krb5kdc[4461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:18.550969 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:18.531888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:48015 (local address 127.0.205.190:39553)
0808 02:57:18.532398 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:18.532402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:18.533907 (+ 1505us) server_negotiation.cc:407] Connection header received
0808 02:57:18.535875 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:18.535880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:18.535973 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:18.536117 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:18.537720 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:18.538599 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:18.540196 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:18.540625 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:18.540774 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:18.544645 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:18.544652 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:18.544656 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:18.544699 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:18.547948 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:18.548712 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:18.548716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:18.548718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:18.548849 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:18.549253 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:18.549256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:18.549258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:18.549534 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:18.549646 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:18.550492 (+ 846us) server_negotiation.cc:299] Negotiation successful
0808 02:57:18.550671 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":269,"threads_started":1}
I20260808 02:57:18.552105 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:18.532727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:39553 (local address 127.0.205.131:48015)
0808 02:57:18.533684 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:18.533759 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:18.535586 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:18.536383 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:18.536395 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:18.536956 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:18.537543 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:18.537560 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:18.538779 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:18.538783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:18.540044 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:18.540056 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:18.540874 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:18.540892 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:18.541142 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:18.541956 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:18.541985 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:18.544355 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:18.548097 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:18.548106 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:18.548121 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:18.548601 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:18.548955 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:18.548958 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:18.548960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:18.549155 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:18.549646 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:18.549652 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:18.550087 (+ 435us) client_negotiation.cc:769] Sending connection context
0808 02:57:18.550389 (+ 302us) client_negotiation.cc:240] Negotiation successful
0808 02:57:18.550815 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":368,"threads_started":1}
I20260808 02:57:18.553571 5189 heartbeater.cc:347] Connected to a master server at 127.0.205.190:39553
I20260808 02:57:18.553973 5189 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:18.554803 5189 heartbeater.cc:511] Master 127.0.205.190:39553 requested a full tablet report, sending...
I20260808 02:57:18.556277 4526 ts_manager.cc:194] Registered new tserver with Master: f29aafe6ebcd487191a5c35c29534dc6 (127.0.205.131:40539)
I20260808 02:57:18.557117 4526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:48015
I20260808 02:57:18.559625 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:57:18.598237 5192 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:57:18.595498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.0.205.190:39553)
0808 02:57:18.595690 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:18.595694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:18.596183 (+ 489us) server_negotiation.cc:407] Connection header received
0808 02:57:18.596339 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:18.596343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:18.596453 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:18.596554 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:18.598033 (+ 1479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49144: BlockingRecv error: recv got EOF from 127.0.0.1:49144 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:57:18.601452 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 4615
W20260808 02:57:18.604416 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 4615: Not authorized: Client connection negotiation failed: client connection to 127.0.205.129:37149: server requires authentication, but client does not have Kerberos credentials available
I20260808 02:57:18.604617 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 4615
I20260808 02:57:18.636597 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 4810
W20260808 02:57:18.639576 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 4810: Not authorized: Client connection negotiation failed: client connection to 127.0.205.130:33421: server requires authentication, but client does not have Kerberos credentials available
I20260808 02:57:18.639766 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 4810
I20260808 02:57:18.664580 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 5006
W20260808 02:57:18.667980 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 5006: Not authorized: Client connection negotiation failed: client connection to 127.0.205.131:40539: server requires authentication, but client does not have Kerberos credentials available
I20260808 02:57:18.668129 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 5006
I20260808 02:57:18.688730 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 4479
I20260808 02:57:18.690137 4571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:18.796756 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 4479
2026-08-08T02:57:18Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4025 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:57:18 dist-test-slave-6ntk krb5kdc[5215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:57:18 dist-test-slave-6ntk krb5kdc[5215](info): set up 2 sockets
Aug 08 02:57:18 dist-test-slave-6ntk krb5kdc[5215](info): commencing operation
krb5kdc: starting...
W20260808 02:57:20.125968 822 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:57:20 dist-test-slave-6ntk krb5kdc[5215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:57:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:57:20.362110 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:36765
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:45289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:36765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:20.701777 5232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:20.702111 5232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:20.702256 5232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:20.709693 5232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:20.709808 5232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:20.709854 5232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:20.709928 5232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:20.709969 5232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:20.719906 5232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:45289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:36765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:36765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:20.721303 5232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:20.722657 5232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:20.735368 5238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:20.735329 5241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:20.735329 5239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:20.736177 5232 server_base.cc:1034] running on GCE node
I20260808 02:57:20.737061 5232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:20.738636 5232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:20.739917 5232 hybrid_clock.cc:648] HybridClock initialized: now 1786157840739844 us; error 67 us; skew 500 ppm
I20260808 02:57:20.740295 5232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:20 dist-test-slave-6ntk krb5kdc[5215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:20.743698 5232 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:57:20.745193 5232 webserver.cc:460] Webserver started at http://127.0.205.190:39057/ using document root <none> and password file <none>
I20260808 02:57:20.746007 5232 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:20.746162 5232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:20.746481 5232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:20.749533 5232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "88e1db4a1698439ba79dcace4a60a225"
format_stamp: "Formatted at 2026-08-08 02:57:20 on dist-test-slave-6ntk"
server_key: "47c2fc0bf965c1148201f99dbde3da5a"
server_key_iv: "dfe5fd2674fc487ea43c21c3dd7cf7ae"
server_key_version: "encryptionkey@0"
I20260808 02:57:20.750447 5232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "88e1db4a1698439ba79dcace4a60a225"
format_stamp: "Formatted at 2026-08-08 02:57:20 on dist-test-slave-6ntk"
server_key: "47c2fc0bf965c1148201f99dbde3da5a"
server_key_iv: "dfe5fd2674fc487ea43c21c3dd7cf7ae"
server_key_version: "encryptionkey@0"
I20260808 02:57:20.755832 5232 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:57:20.759461 5249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:20.760802 5232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:57:20.761030 5232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "88e1db4a1698439ba79dcace4a60a225"
format_stamp: "Formatted at 2026-08-08 02:57:20 on dist-test-slave-6ntk"
server_key: "47c2fc0bf965c1148201f99dbde3da5a"
server_key_iv: "dfe5fd2674fc487ea43c21c3dd7cf7ae"
server_key_version: "encryptionkey@0"
I20260808 02:57:20.761243 5232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:20.774084 5232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:20.778609 5232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:20.778923 5232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:20.797570 5232 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:36765
I20260808 02:57:20.797605 5325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:36765 every 8 connection(s)
I20260808 02:57:20.799165 5232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:20.803009 5326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:20.804667 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 5232
I20260808 02:57:20.804895 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:20.805743 822 external_mini_cluster.cc:1428] Setting key 6de8d621d34feb3ea82bd3b797c9f070
I20260808 02:57:20.818095 5326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225: Bootstrap starting.
Aug 08 02:57:20 dist-test-slave-6ntk krb5kdc[5215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:20.821990 5326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:20.823089 5326 log.cc:824] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:20.826016 5326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225: No bootstrap required, opened a new log
I20260808 02:57:20.829515 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:20.807711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48106 (local address 127.0.205.190:36765)
0808 02:57:20.808589 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:20.808604 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:20.808652 (+ 48us) server_negotiation.cc:407] Connection header received
0808 02:57:20.810308 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:20.810335 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:20.810751 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:20.811078 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:20.812936 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:20.814341 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:20.816246 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:20.816945 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:20.817078 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:20.821855 (+ 4777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:20.821863 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:20.821925 (+ 62us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:20.821977 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:20.825153 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:20.825931 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:20.825942 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:20.825957 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:20.826183 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:20.826598 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:20.826604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:20.826607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:20.827170 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:20.827513 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:20.828053 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 02:57:20.828485 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":438,"threads_started":1}
I20260808 02:57:20.833822 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:57:20.835053 5326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e1db4a1698439ba79dcace4a60a225" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36765 } }
I20260808 02:57:20.835438 5326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:20.835567 5326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e1db4a1698439ba79dcace4a60a225, State: Initialized, Role: FOLLOWER
I20260808 02:57:20.836143 5326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e1db4a1698439ba79dcace4a60a225" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36765 } }
I20260808 02:57:20.836373 5326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:20.836510 5326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:20.836697 5326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:20.839121 5326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e1db4a1698439ba79dcace4a60a225" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36765 } }
I20260808 02:57:20.839681 5326 leader_election.cc:302] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88e1db4a1698439ba79dcace4a60a225; no voters:
I20260808 02:57:20.840418 5326 leader_election.cc:288] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:20.840489 5331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:20.841777 5331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [term 1 LEADER]: Becoming Leader. State: Replica: 88e1db4a1698439ba79dcace4a60a225, State: Running, Role: LEADER
I20260808 02:57:20.842470 5331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e1db4a1698439ba79dcace4a60a225" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36765 } }
I20260808 02:57:20.843021 5326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:20.846196 5333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88e1db4a1698439ba79dcace4a60a225. Latest consensus state: current_term: 1 leader_uuid: "88e1db4a1698439ba79dcace4a60a225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e1db4a1698439ba79dcace4a60a225" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36765 } } }
I20260808 02:57:20.846225 5332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88e1db4a1698439ba79dcace4a60a225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e1db4a1698439ba79dcace4a60a225" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36765 } } }
I20260808 02:57:20.846662 5333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:20.846668 5332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:20.852154 5338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:20.859362 5338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:20.882822 5338 catalog_manager.cc:1347] Generated new cluster ID: dd5818ddf7d146a0b77c1f831e4d2d45
I20260808 02:57:20.882975 5338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:20.918236 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:20.904804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48114 (local address 127.0.205.190:36765)
0808 02:57:20.904988 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:20.904994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:20.908768 (+ 3774us) server_negotiation.cc:407] Connection header received
0808 02:57:20.908966 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:20.908972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:20.909053 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:20.909163 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:20.910611 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:20.911712 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:20.913676 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:20.914370 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:20.914484 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:20.916039 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:20.916046 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:20.916051 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:20.917220 (+ 1169us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:20.917301 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:20.917338 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:20.917454 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:20.917723 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 02:57:20.917824 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":60}
I20260808 02:57:20.921635 5338 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:20.923535 5338 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:20.940173 5338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88e1db4a1698439ba79dcace4a60a225: Generated new TSK 0
I20260808 02:57:20.941402 5338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:57:20.954809 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 5232
I20260808 02:57:20.956033 5320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:21.050031 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 5232
2026-08-08T02:57:21Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2250 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:57:21 dist-test-slave-6ntk krb5kdc[5371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:57:21 dist-test-slave-6ntk krb5kdc[5371](info): set up 2 sockets
Aug 08 02:57:21 dist-test-slave-6ntk krb5kdc[5371](info): commencing operation
krb5kdc: starting...
W20260808 02:57:22.390357 822 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:57:22 dist-test-slave-6ntk krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:57:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:57:22.619637 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:38701
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:38701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:22.954277 5386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:22.954671 5386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:22.954849 5386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:22.962877 5386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:22.963034 5386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:22.963147 5386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:22.963258 5386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:22.963335 5386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:22.973250 5386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:38701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:38701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:22.974738 5386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:22.976094 5386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:22.988556 5393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:22.988595 5394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:22.989084 5396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:22.989311 5386 server_base.cc:1034] running on GCE node
I20260808 02:57:22.990495 5386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:22.991956 5386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:22.993170 5386 hybrid_clock.cc:648] HybridClock initialized: now 1786157842993123 us; error 26 us; skew 500 ppm
I20260808 02:57:22.993511 5386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:22 dist-test-slave-6ntk krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:23.002084 5386 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:57:23.003705 5386 webserver.cc:460] Webserver started at http://127.0.205.190:44717/ using document root <none> and password file <none>
I20260808 02:57:23.004451 5386 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:23.004613 5386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:23.004941 5386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:23.008195 5386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "665e7590270e4a00b837967e303ef8a9"
format_stamp: "Formatted at 2026-08-08 02:57:23 on dist-test-slave-6ntk"
server_key: "b3b00913f141df3b9dccbc936423d371"
server_key_iv: "46edfbb6dcd0e12dfa85b234db232f1c"
server_key_version: "encryptionkey@0"
I20260808 02:57:23.009027 5386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "665e7590270e4a00b837967e303ef8a9"
format_stamp: "Formatted at 2026-08-08 02:57:23 on dist-test-slave-6ntk"
server_key: "b3b00913f141df3b9dccbc936423d371"
server_key_iv: "46edfbb6dcd0e12dfa85b234db232f1c"
server_key_version: "encryptionkey@0"
I20260808 02:57:23.014811 5386 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 02:57:23.018463 5407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:23.019837 5386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:57:23.020042 5386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "665e7590270e4a00b837967e303ef8a9"
format_stamp: "Formatted at 2026-08-08 02:57:23 on dist-test-slave-6ntk"
server_key: "b3b00913f141df3b9dccbc936423d371"
server_key_iv: "46edfbb6dcd0e12dfa85b234db232f1c"
server_key_version: "encryptionkey@0"
I20260808 02:57:23.020255 5386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:23.056142 5386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:23.060678 5386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:23.061005 5386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:23.080509 5386 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:38701
I20260808 02:57:23.080557 5472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:38701 every 8 connection(s)
I20260808 02:57:23.082085 5386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:23.083000 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 5386
I20260808 02:57:23.083284 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:23.083925 822 external_mini_cluster.cc:1428] Setting key 999a2339db6bf511b7e696b94e09f95b
I20260808 02:57:23.086640 5473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:57:23 dist-test-slave-6ntk krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:23.100544 5473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9: Bootstrap starting.
I20260808 02:57:23.104945 5473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:23.106181 5473 log.cc:824] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:23.108912 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:23.086016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38616 (local address 127.0.205.190:38701)
0808 02:57:23.086901 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:23.086912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:23.086960 (+ 48us) server_negotiation.cc:407] Connection header received
0808 02:57:23.088602 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:23.088626 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:23.089030 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:23.089378 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:23.091268 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:23.092692 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:23.094574 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:23.095279 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:23.095431 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:23.100276 (+ 4845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:23.100290 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:23.100310 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:23.100357 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:23.104403 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:23.105234 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:23.105242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:23.105248 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:23.105452 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:23.105914 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:23.105921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:23.105925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:23.106553 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:23.106985 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:23.107477 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 02:57:23.107931 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":382,"threads_started":1}
I20260808 02:57:23.109510 5473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9: No bootstrap required, opened a new log
I20260808 02:57:23.118297 5473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665e7590270e4a00b837967e303ef8a9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38701 } }
I20260808 02:57:23.118722 5473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:23.118839 5473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 665e7590270e4a00b837967e303ef8a9, State: Initialized, Role: FOLLOWER
I20260808 02:57:23.119566 5473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665e7590270e4a00b837967e303ef8a9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38701 } }
I20260808 02:57:23.119855 5473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:23.120025 5473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:23.120224 5473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:23.123600 5473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665e7590270e4a00b837967e303ef8a9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38701 } }
I20260808 02:57:23.124204 5473 leader_election.cc:302] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 665e7590270e4a00b837967e303ef8a9; no voters:
I20260808 02:57:23.124948 5473 leader_election.cc:288] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:23.125064 5482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:23.126163 5482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [term 1 LEADER]: Becoming Leader. State: Replica: 665e7590270e4a00b837967e303ef8a9, State: Running, Role: LEADER
I20260808 02:57:23.126699 5482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665e7590270e4a00b837967e303ef8a9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38701 } }
I20260808 02:57:23.127305 5473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:23.129930 5483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "665e7590270e4a00b837967e303ef8a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665e7590270e4a00b837967e303ef8a9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38701 } } }
I20260808 02:57:23.129930 5484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 665e7590270e4a00b837967e303ef8a9. Latest consensus state: current_term: 1 leader_uuid: "665e7590270e4a00b837967e303ef8a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "665e7590270e4a00b837967e303ef8a9" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38701 } } }
I20260808 02:57:23.130385 5483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:23.130385 5484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:23.133395 5489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:23.139212 5489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:23.148577 5489 catalog_manager.cc:1347] Generated new cluster ID: 3f93861bd16f47f9bec04da4f81166ce
I20260808 02:57:23.148715 5489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:23.171948 5489 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:23.173346 5489 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:23.184129 5489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 665e7590270e4a00b837967e303ef8a9: Generated new TSK 0
I20260808 02:57:23.185034 5489 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:57:23.258194 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38701
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:23.637758 5511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:23.638134 5511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:23.638307 5511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:23.645759 5511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:23.645998 5511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:23.646219 5511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:23.656466 5511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:23.657852 5511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:23.659334 5511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:23.674012 5519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:23.674235 5518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:23.674238 5521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:23.679481 5511 server_base.cc:1034] running on GCE node
I20260808 02:57:23.680153 5511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:23.681186 5511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:23.682431 5511 hybrid_clock.cc:648] HybridClock initialized: now 1786157843682386 us; error 41 us; skew 500 ppm
I20260808 02:57:23.682761 5511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:23 dist-test-slave-6ntk krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:23.686187 5511 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:57:23.687871 5511 webserver.cc:460] Webserver started at http://127.0.205.129:32975/ using document root <none> and password file <none>
I20260808 02:57:23.688635 5511 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:23.688787 5511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:23.689108 5511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:23.692179 5511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "f8a269a3faa3416c84a8edfef709f5b9"
format_stamp: "Formatted at 2026-08-08 02:57:23 on dist-test-slave-6ntk"
server_key: "b7b540b90cc67f34e0cc2afe5c05a1de"
server_key_iv: "bdc8ddc7d53dc773178f260f66ca3a56"
server_key_version: "encryptionkey@0"
I20260808 02:57:23.693004 5511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "f8a269a3faa3416c84a8edfef709f5b9"
format_stamp: "Formatted at 2026-08-08 02:57:23 on dist-test-slave-6ntk"
server_key: "b7b540b90cc67f34e0cc2afe5c05a1de"
server_key_iv: "bdc8ddc7d53dc773178f260f66ca3a56"
server_key_version: "encryptionkey@0"
I20260808 02:57:23.698349 5511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:57:23.702051 5535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:23.703467 5511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:57:23.703657 5511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "f8a269a3faa3416c84a8edfef709f5b9"
format_stamp: "Formatted at 2026-08-08 02:57:23 on dist-test-slave-6ntk"
server_key: "b7b540b90cc67f34e0cc2afe5c05a1de"
server_key_iv: "bdc8ddc7d53dc773178f260f66ca3a56"
server_key_version: "encryptionkey@0"
I20260808 02:57:23.703859 5511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:23.724169 5511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:23.728952 5511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:23.729260 5511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:23.730280 5511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:23.731752 5511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:23.731861 5511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:23.732013 5511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:23.732087 5511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:23.768541 5511 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:33469
I20260808 02:57:23.768592 5690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:33469 every 8 connection(s)
I20260808 02:57:23.769972 5511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:23.775141 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 5511
I20260808 02:57:23.775353 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:57:23.776041 822 external_mini_cluster.cc:1428] Setting key 9d9f6a9326ec551ecae600d4762f8bf4
Aug 08 02:57:23 dist-test-slave-6ntk krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:23.792873 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:23.772700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:54731 (local address 127.0.205.190:38701)
0808 02:57:23.773180 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:23.773185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:23.774085 (+ 900us) server_negotiation.cc:407] Connection header received
0808 02:57:23.776137 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:23.776143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:23.776242 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:23.776373 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:23.778541 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:23.779249 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:23.780888 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:23.781287 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:23.781433 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:23.785560 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:23.785569 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:23.785577 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:23.785626 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:23.789270 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:23.790136 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:23.790144 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:23.790150 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:23.790322 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:23.790763 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:23.790770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:23.790776 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:23.791178 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:23.791354 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:23.792119 (+ 765us) server_negotiation.cc:299] Negotiation successful
0808 02:57:23.792380 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":272,"threads_started":1}
I20260808 02:57:23.793543 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:23.773147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:38701 (local address 127.0.205.129:54731)
0808 02:57:23.773852 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:23.773968 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:23.775846 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:23.776677 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:23.776689 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:23.777434 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:23.778356 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:23.778379 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:23.779425 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:23.779431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:23.780701 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:23.780713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:23.781542 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:23.781548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:23.781790 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:23.782585 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:23.782608 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:23.785300 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:23.789448 (+ 4148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:23.789456 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:23.789470 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:23.789995 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:23.790450 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:23.790453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:23.790456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:23.790641 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:23.791326 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:23.791332 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:23.791696 (+ 364us) client_negotiation.cc:769] Sending connection context
0808 02:57:23.791941 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 02:57:23.792361 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":271,"threads_started":1}
I20260808 02:57:23.795305 5691 heartbeater.cc:347] Connected to a master server at 127.0.205.190:38701
I20260808 02:57:23.795672 5691 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:23.796669 5691 heartbeater.cc:511] Master 127.0.205.190:38701 requested a full tablet report, sending...
I20260808 02:57:23.799157 5425 ts_manager.cc:194] Registered new tserver with Master: f8a269a3faa3416c84a8edfef709f5b9 (127.0.205.129:33469)
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
I20260808 02:57:23.800733 5425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:54731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:57:23.831380 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38701
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:24.197577 5699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:24.197912 5699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:24.198053 5699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:24.205641 5699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:24.205758 5699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:24.205950 5699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:24.216478 5699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:24.217800 5699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:24.219203 5699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:24.232806 5706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:24.232752 5712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:24.232720 5708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:24.234125 5699 server_base.cc:1034] running on GCE node
I20260808 02:57:24.234913 5699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:24.235857 5699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:24.237110 5699 hybrid_clock.cc:648] HybridClock initialized: now 1786157844237056 us; error 55 us; skew 500 ppm
I20260808 02:57:24.237447 5699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:24 dist-test-slave-6ntk krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:24.240705 5699 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:57:24.242352 5699 webserver.cc:460] Webserver started at http://127.0.205.130:34075/ using document root <none> and password file <none>
I20260808 02:57:24.243067 5699 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:24.243218 5699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:24.243548 5699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:24.246625 5699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "3ec3772377ae4c069e15bde308de2323"
format_stamp: "Formatted at 2026-08-08 02:57:24 on dist-test-slave-6ntk"
server_key: "c22b355da37131829ecc979eb4359a4a"
server_key_iv: "a51330f7d7545040ad668d2b586464ab"
server_key_version: "encryptionkey@0"
I20260808 02:57:24.247452 5699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "3ec3772377ae4c069e15bde308de2323"
format_stamp: "Formatted at 2026-08-08 02:57:24 on dist-test-slave-6ntk"
server_key: "c22b355da37131829ecc979eb4359a4a"
server_key_iv: "a51330f7d7545040ad668d2b586464ab"
server_key_version: "encryptionkey@0"
I20260808 02:57:24.252767 5699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:57:24.256351 5723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:24.257750 5699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 02:57:24.257964 5699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "3ec3772377ae4c069e15bde308de2323"
format_stamp: "Formatted at 2026-08-08 02:57:24 on dist-test-slave-6ntk"
server_key: "c22b355da37131829ecc979eb4359a4a"
server_key_iv: "a51330f7d7545040ad668d2b586464ab"
server_key_version: "encryptionkey@0"
I20260808 02:57:24.258190 5699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:24.314599 5699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:24.318986 5699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:24.319314 5699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:24.320350 5699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:24.321725 5699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:24.321833 5699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:24.322017 5699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:24.322095 5699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:24.357771 5699 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:35459
I20260808 02:57:24.357882 5868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:35459 every 8 connection(s)
I20260808 02:57:24.359208 5699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:24.366353 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 5699
I20260808 02:57:24.366556 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:57:24.367159 822 external_mini_cluster.cc:1428] Setting key e8011f77895b1ba8b4e6bdb49e1fb060
Aug 08 02:57:24 dist-test-slave-6ntk krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:24.383245 5871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:24.361969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:47331 (local address 127.0.205.190:38701)
0808 02:57:24.362511 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:24.362515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:24.363871 (+ 1356us) server_negotiation.cc:407] Connection header received
0808 02:57:24.366191 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:24.366196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:24.366266 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:24.366361 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:24.368419 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.369535 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.372114 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.372544 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.372641 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:24.377165 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:24.377170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:24.377173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:24.377206 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:24.380038 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:24.380824 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:24.380828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:24.380831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:24.380960 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:24.381425 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:24.381429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:24.381431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:24.381705 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:24.381872 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:24.382623 (+ 751us) server_negotiation.cc:299] Negotiation successful
0808 02:57:24.382847 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":282,"threads_started":1}
I20260808 02:57:24.384303 5872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:24.362687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:38701 (local address 127.0.205.130:47331)
0808 02:57:24.363607 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:24.363657 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:24.365792 (+ 2135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:24.366664 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:24.366674 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:24.367441 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:24.368230 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:24.368261 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.369702 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.369710 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:24.371952 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:24.371972 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.372780 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.372790 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:24.373127 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:24.373935 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:24.373972 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:24.376909 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:24.380199 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:24.380208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:24.380226 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:24.380680 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:24.381076 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:24.381081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:24.381085 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:24.381284 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:24.381850 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:24.381858 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:24.382248 (+ 390us) client_negotiation.cc:769] Sending connection context
0808 02:57:24.382526 (+ 278us) client_negotiation.cc:240] Negotiation successful
0808 02:57:24.383131 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":329,"threads_started":1}
I20260808 02:57:24.385921 5869 heartbeater.cc:347] Connected to a master server at 127.0.205.190:38701
I20260808 02:57:24.386304 5869 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:24.387328 5869 heartbeater.cc:511] Master 127.0.205.190:38701 requested a full tablet report, sending...
I20260808 02:57:24.389032 5425 ts_manager.cc:194] Registered new tserver with Master: 3ec3772377ae4c069e15bde308de2323 (127.0.205.130:35459)
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
I20260808 02:57:24.389992 5425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:47331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:57:24.422295 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38701
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:24.768118 5877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:24.768452 5877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:24.768620 5877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:24.776257 5877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:24.776368 5877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:24.776578 5877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:24.786756 5877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:24.788070 5877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:24.789376 5877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:24.802889 5884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:24.802814 5892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:24.803067 5885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:24.804044 5691 heartbeater.cc:503] Master 127.0.205.190:38701 was elected leader, sending a full tablet report...
I20260808 02:57:24.804499 5877 server_base.cc:1034] running on GCE node
I20260808 02:57:24.805238 5877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:24.806496 5877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:24.807750 5877 hybrid_clock.cc:648] HybridClock initialized: now 1786157844807699 us; error 44 us; skew 500 ppm
I20260808 02:57:24.808105 5877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:24 dist-test-slave-6ntk krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:24.811465 5877 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:57:24.812944 5877 webserver.cc:460] Webserver started at http://127.0.205.131:32961/ using document root <none> and password file <none>
I20260808 02:57:24.814072 5877 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:24.814338 5877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:24.814802 5877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:24.818425 5877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "d68b44d2400d4d439f2a67cd6bb32f08"
format_stamp: "Formatted at 2026-08-08 02:57:24 on dist-test-slave-6ntk"
server_key: "4c8d1bd21b0fec7e70e17c8d39b9dee6"
server_key_iv: "13218e62385c8fcdf33a25ab4014397b"
server_key_version: "encryptionkey@0"
I20260808 02:57:24.819389 5877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "d68b44d2400d4d439f2a67cd6bb32f08"
format_stamp: "Formatted at 2026-08-08 02:57:24 on dist-test-slave-6ntk"
server_key: "4c8d1bd21b0fec7e70e17c8d39b9dee6"
server_key_iv: "13218e62385c8fcdf33a25ab4014397b"
server_key_version: "encryptionkey@0"
I20260808 02:57:24.825595 5877 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260808 02:57:24.829463 5905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:24.830925 5877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:24.831110 5877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "d68b44d2400d4d439f2a67cd6bb32f08"
format_stamp: "Formatted at 2026-08-08 02:57:24 on dist-test-slave-6ntk"
server_key: "4c8d1bd21b0fec7e70e17c8d39b9dee6"
server_key_iv: "13218e62385c8fcdf33a25ab4014397b"
server_key_version: "encryptionkey@0"
I20260808 02:57:24.831329 5877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:24.851714 5877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:24.856098 5877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:24.856429 5877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:24.857432 5877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:24.858902 5877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:24.859018 5877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:24.859175 5877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:24.859251 5877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:24.894068 5877 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:33015
I20260808 02:57:24.894106 6071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:33015 every 8 connection(s)
I20260808 02:57:24.895555 5877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:57:24.896950 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 5877
I20260808 02:57:24.897146 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:57:24.897734 822 external_mini_cluster.cc:1428] Setting key 66a731f83125c6545acb56a71393f4cc
Aug 08 02:57:24 dist-test-slave-6ntk krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:24.918020 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:24.899006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:60631 (local address 127.0.205.190:38701)
0808 02:57:24.899486 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:24.899493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:24.900440 (+ 947us) server_negotiation.cc:407] Connection header received
0808 02:57:24.902522 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:24.902527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:24.902632 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:24.902764 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:24.904374 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.905180 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.906911 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.907405 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.907545 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:24.911296 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:24.911301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:24.911304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:24.911346 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:24.914545 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:24.915570 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:24.915575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:24.915577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:24.915700 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:24.916105 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:24.916108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:24.916110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:24.916378 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:24.916506 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:24.917232 (+ 726us) server_negotiation.cc:299] Negotiation successful
0808 02:57:24.917647 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":235,"threads_started":1}
I20260808 02:57:24.918802 6075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:24.899304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:38701 (local address 127.0.205.131:60631)
0808 02:57:24.900265 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:24.900312 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:24.902210 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:24.903033 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:24.903043 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:24.903588 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:24.904186 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:24.904205 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.905357 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.905361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:24.906753 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:24.906766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.907665 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.907672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:24.907935 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:24.908674 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:24.908703 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:24.911071 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:24.914705 (+ 3634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:24.914719 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:24.914733 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:24.915418 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:24.915797 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:24.915800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:24.915802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:24.915999 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:24.916528 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:24.916535 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:24.916878 (+ 343us) client_negotiation.cc:769] Sending connection context
0808 02:57:24.917133 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 02:57:24.917574 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":372,"threads_started":1}
I20260808 02:57:24.920251 6072 heartbeater.cc:347] Connected to a master server at 127.0.205.190:38701
I20260808 02:57:24.920547 6072 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:24.921260 6072 heartbeater.cc:511] Master 127.0.205.190:38701 requested a full tablet report, sending...
I20260808 02:57:24.922772 5425 ts_manager.cc:194] Registered new tserver with Master: d68b44d2400d4d439f2a67cd6bb32f08 (127.0.205.131:33015)
I20260808 02:57:24.923561 5425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:60631
I20260808 02:57:24.926962 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:57:24 dist-test-slave-6ntk krb5kdc[5371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157844, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 02:57:24 dist-test-slave-6ntk krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157844, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:24.981961 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:24.967991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.0.205.190:38701)
0808 02:57:24.968185 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:24.968190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:24.968665 (+ 475us) server_negotiation.cc:407] Connection header received
0808 02:57:24.968824 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:24.968828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:24.968910 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:24.969007 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:24.970522 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.971246 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.972432 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.972812 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.972900 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:24.977155 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:24.977160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:24.977163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:24.977197 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:24.979708 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:24.980306 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:24.980309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:24.980312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:24.980439 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:24.980757 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:24.980760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:24.980762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:24.981074 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:24.981249 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:24.981507 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 02:57:24.981666 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 02:57:24.982529 5425 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38632
I20260808 02:57:24.984707 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 5511
Aug 08 02:57:24 dist-test-slave-6ntk krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157844, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:25.000741 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:24.985809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52518 (local address 127.0.205.129:33469)
0808 02:57:24.986508 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:24.986515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:24.986560 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:57:24.986641 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:24.986652 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:24.986961 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:24.987122 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:24.988220 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.989252 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.990469 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:24.991041 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:24.991145 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:24.995155 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:24.995168 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:24.995182 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:24.995216 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:24.998251 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:24.998857 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:24.998863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:24.998866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:24.998985 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:24.999312 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:24.999316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:24.999318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:24.999668 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:24.999899 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:25.000130 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 02:57:25.000324 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":380,"threads_started":1}
W20260808 02:57:25.002077 5685 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52518
W20260808 02:57:25.003175 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 5511: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:57:25.003409 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 5511
I20260808 02:57:25.033388 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 5699
Aug 08 02:57:25 dist-test-slave-6ntk krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157844, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:57:25.050721 6086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:25.034491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32944 (local address 127.0.205.130:35459)
0808 02:57:25.035103 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:25.035131 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:25.035157 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:57:25.035220 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:25.035226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:25.035505 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:25.035717 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:25.036924 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:25.038051 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:25.039346 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:25.039932 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:25.040075 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:25.044803 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:25.044820 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:25.044845 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:25.044882 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:25.047712 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:25.048450 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:25.048458 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:25.048460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:25.048591 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:25.049000 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:25.049007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:25.049010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:25.049373 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:25.049713 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:25.050076 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 02:57:25.050358 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":333,"threads_started":1}
W20260808 02:57:25.052069 5860 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32944
W20260808 02:57:25.053208 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 5699: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:57:25.053349 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 5699
I20260808 02:57:25.085649 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 5877
Aug 08 02:57:25 dist-test-slave-6ntk krb5kdc[5371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157844, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:57:25.102624 6087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:25.086832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37418 (local address 127.0.205.131:33015)
0808 02:57:25.087523 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:25.087529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:25.087557 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:57:25.087631 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:25.087638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:25.087851 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:25.088011 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:25.089346 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:25.090478 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:25.091715 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:25.092279 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:25.092375 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:25.096855 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:25.096867 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:25.096882 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:25.096937 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:25.099860 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:25.100549 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:25.100555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:25.100558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:25.100690 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:25.101012 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:25.101015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:25.101017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:25.101369 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:25.101616 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:25.101858 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 02:57:25.102193 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":352,"threads_started":1}
W20260808 02:57:25.104005 6065 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37418
W20260808 02:57:25.105248 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 5877: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 02:57:25.105458 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 5877
I20260808 02:57:25.126080 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 5386
I20260808 02:57:25.127319 5467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:25.235146 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 5386
2026-08-08T02:57:25Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4187 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:57:25 dist-test-slave-6ntk krb5kdc[6092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:57:25 dist-test-slave-6ntk krb5kdc[6092](info): set up 2 sockets
Aug 08 02:57:25 dist-test-slave-6ntk krb5kdc[6092](info): commencing operation
krb5kdc: starting...
W20260808 02:57:26.560762 822 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:57:26 dist-test-slave-6ntk krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:57:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:57:26.797979 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:42123
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:42123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:27.139511 6112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:27.139813 6112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:27.139961 6112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:27.147466 6112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:27.147581 6112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:27.147686 6112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:27.147796 6112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:27.147881 6112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:27.158099 6112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:42123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:42123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:27.159518 6112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:27.160912 6112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:27.173360 6123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:27.173384 6120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:27.173381 6117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:27.174172 6112 server_base.cc:1034] running on GCE node
I20260808 02:57:27.175046 6112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:27.176658 6112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:27.177975 6112 hybrid_clock.cc:648] HybridClock initialized: now 1786157847177910 us; error 77 us; skew 500 ppm
I20260808 02:57:27.178351 6112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:27 dist-test-slave-6ntk krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:27.181726 6112 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:57:27.183269 6112 webserver.cc:460] Webserver started at http://127.0.205.190:46273/ using document root <none> and password file <none>
I20260808 02:57:27.183951 6112 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:27.184085 6112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:27.184370 6112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:27.187613 6112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "13cd4ee408114aaa9f4b553501800611"
format_stamp: "Formatted at 2026-08-08 02:57:27 on dist-test-slave-6ntk"
server_key: "57af8043036d1570a3a6e5de7c31216a"
server_key_iv: "cd0c69d04b640afb3f5f00d7d3385dbf"
server_key_version: "encryptionkey@0"
I20260808 02:57:27.188503 6112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "13cd4ee408114aaa9f4b553501800611"
format_stamp: "Formatted at 2026-08-08 02:57:27 on dist-test-slave-6ntk"
server_key: "57af8043036d1570a3a6e5de7c31216a"
server_key_iv: "cd0c69d04b640afb3f5f00d7d3385dbf"
server_key_version: "encryptionkey@0"
I20260808 02:57:27.194049 6112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 02:57:27.197539 6132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:27.198997 6112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:27.199190 6112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "13cd4ee408114aaa9f4b553501800611"
format_stamp: "Formatted at 2026-08-08 02:57:27 on dist-test-slave-6ntk"
server_key: "57af8043036d1570a3a6e5de7c31216a"
server_key_iv: "cd0c69d04b640afb3f5f00d7d3385dbf"
server_key_version: "encryptionkey@0"
I20260808 02:57:27.199409 6112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:27.230064 6112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:27.234375 6112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:27.234704 6112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:27.254320 6112 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:42123
I20260808 02:57:27.254341 6207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:42123 every 8 connection(s)
I20260808 02:57:27.255853 6112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:27.260107 6209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:27.262068 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 6112
I20260808 02:57:27.262327 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:27.263248 822 external_mini_cluster.cc:1428] Setting key 7d85aa6929473f5a898ccff4561b0b40
I20260808 02:57:27.273775 6209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611: Bootstrap starting.
I20260808 02:57:27.277969 6209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611: Neither blocks nor log segments found. Creating new log.
Aug 08 02:57:27 dist-test-slave-6ntk krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:27.279211 6209 log.cc:824] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:27.282902 6209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611: No bootstrap required, opened a new log
I20260808 02:57:27.289036 6214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:27.265451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35260 (local address 127.0.205.190:42123)
0808 02:57:27.266407 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:27.266420 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:27.266464 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:57:27.268286 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:27.268313 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:27.268747 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:27.269080 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:27.271096 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:27.272482 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:27.274259 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:27.274942 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:27.275078 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:27.280054 (+ 4976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:27.280065 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:27.280081 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:27.280125 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:27.284177 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:27.284921 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:27.284930 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:27.284941 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:27.285175 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:27.285555 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:27.285560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:27.285564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:27.286414 (+ 850us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:27.286731 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:27.287109 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 02:57:27.287608 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":454,"threads_started":1}
I20260808 02:57:27.293362 6209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd4ee408114aaa9f4b553501800611" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42123 } }
I20260808 02:57:27.293717 6209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:27.293797 6209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13cd4ee408114aaa9f4b553501800611, State: Initialized, Role: FOLLOWER
I20260808 02:57:27.294430 6209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd4ee408114aaa9f4b553501800611" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42123 } }
I20260808 02:57:27.294677 6209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:27.294816 6209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:27.295019 6209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:27.297307 6209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd4ee408114aaa9f4b553501800611" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42123 } }
I20260808 02:57:27.297860 6209 leader_election.cc:302] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13cd4ee408114aaa9f4b553501800611; no voters:
I20260808 02:57:27.298532 6209 leader_election.cc:288] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:27.298681 6216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:27.299414 6216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [term 1 LEADER]: Becoming Leader. State: Replica: 13cd4ee408114aaa9f4b553501800611, State: Running, Role: LEADER
I20260808 02:57:27.299908 6216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd4ee408114aaa9f4b553501800611" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42123 } }
I20260808 02:57:27.300675 6209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:27.302884 6217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13cd4ee408114aaa9f4b553501800611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd4ee408114aaa9f4b553501800611" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42123 } } }
I20260808 02:57:27.303419 6217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:27.303419 6218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13cd4ee408114aaa9f4b553501800611. Latest consensus state: current_term: 1 leader_uuid: "13cd4ee408114aaa9f4b553501800611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13cd4ee408114aaa9f4b553501800611" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42123 } } }
I20260808 02:57:27.303817 6218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:27.306262 6224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:27.312013 6224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:27.321478 6224 catalog_manager.cc:1347] Generated new cluster ID: 7571c30d6f7f48a6b74d94591fe0612d
I20260808 02:57:27.321606 6224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:27.340046 6224 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:27.342003 6224 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:27.369165 6224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13cd4ee408114aaa9f4b553501800611: Generated new TSK 0
I20260808 02:57:27.370368 6224 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:57:27.439870 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:42123
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:27.794996 6252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:27.795375 6252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:27.795547 6252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:27.803445 6252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:27.803639 6252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:27.803831 6252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:27.814790 6252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:42123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:27.816234 6252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:27.817684 6252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:27.832473 6260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:27.832513 6266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:27.832554 6263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:27.833256 6252 server_base.cc:1034] running on GCE node
I20260808 02:57:27.833993 6252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:27.834985 6252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:27.836187 6252 hybrid_clock.cc:648] HybridClock initialized: now 1786157847836123 us; error 57 us; skew 500 ppm
I20260808 02:57:27.836520 6252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:27 dist-test-slave-6ntk krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:27.839787 6252 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:57:27.841238 6252 webserver.cc:460] Webserver started at http://127.0.205.129:41315/ using document root <none> and password file <none>
I20260808 02:57:27.842023 6252 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:27.842175 6252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:27.842514 6252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:27.845631 6252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "88b7a34f13ce47c398320aa26ea4b6db"
format_stamp: "Formatted at 2026-08-08 02:57:27 on dist-test-slave-6ntk"
server_key: "d7faa96b86f0e20ff6fbd5d75612de88"
server_key_iv: "2a6946a6a782bda68742d5dd986be133"
server_key_version: "encryptionkey@0"
I20260808 02:57:27.846657 6252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "88b7a34f13ce47c398320aa26ea4b6db"
format_stamp: "Formatted at 2026-08-08 02:57:27 on dist-test-slave-6ntk"
server_key: "d7faa96b86f0e20ff6fbd5d75612de88"
server_key_iv: "2a6946a6a782bda68742d5dd986be133"
server_key_version: "encryptionkey@0"
I20260808 02:57:27.852303 6252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:57:27.856178 6278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:27.857604 6252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:57:27.857821 6252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "88b7a34f13ce47c398320aa26ea4b6db"
format_stamp: "Formatted at 2026-08-08 02:57:27 on dist-test-slave-6ntk"
server_key: "d7faa96b86f0e20ff6fbd5d75612de88"
server_key_iv: "2a6946a6a782bda68742d5dd986be133"
server_key_version: "encryptionkey@0"
I20260808 02:57:27.858125 6252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:27.880060 6252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:27.884578 6252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:27.884892 6252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:27.885959 6252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:27.887409 6252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:27.887529 6252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:27.887678 6252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:27.887755 6252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:27.922883 6252 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:46095
I20260808 02:57:27.922977 6431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:46095 every 8 connection(s)
I20260808 02:57:27.924407 6252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:27.925479 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 6252
I20260808 02:57:27.925714 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:57:27.926491 822 external_mini_cluster.cc:1428] Setting key fdd08341acdac825dcd1fffd7c38f4a2
Aug 08 02:57:27 dist-test-slave-6ntk krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:27.947000 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:27.927326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:58373 (local address 127.0.205.190:42123)
0808 02:57:27.927865 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:27.927870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:27.929056 (+ 1186us) server_negotiation.cc:407] Connection header received
0808 02:57:27.930987 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:27.930993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:27.931082 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:27.931214 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:27.932762 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:27.933513 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:27.935194 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:27.935605 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:27.935747 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:27.939401 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:27.939410 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:27.939418 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:27.939465 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:27.943135 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:27.944002 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:27.944010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:27.944016 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:27.944173 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:27.944683 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:27.944691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:27.944697 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:27.945162 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:27.945325 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:27.946202 (+ 877us) server_negotiation.cc:299] Negotiation successful
0808 02:57:27.946485 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":287,"threads_started":1}
I20260808 02:57:27.947794 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:27.927960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:42123 (local address 127.0.205.129:58373)
0808 02:57:27.928906 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:27.928951 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:27.930708 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:27.931460 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:27.931468 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:27.931993 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:27.932576 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:27.932599 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:27.933687 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:27.933691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:27.935031 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:27.935043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:27.935857 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:27.935866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:27.936118 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:27.936838 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:27.936861 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:27.939166 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:27.943312 (+ 4146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:27.943320 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:27.943334 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:27.943836 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:27.944322 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:27.944326 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:27.944328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:27.944537 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:27.945327 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:27.945333 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:27.945705 (+ 372us) client_negotiation.cc:769] Sending connection context
0808 02:57:27.946036 (+ 331us) client_negotiation.cc:240] Negotiation successful
0808 02:57:27.946483 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":368,"threads_started":1}
I20260808 02:57:27.949687 6432 heartbeater.cc:347] Connected to a master server at 127.0.205.190:42123
I20260808 02:57:27.950109 6432 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:27.951130 6432 heartbeater.cc:511] Master 127.0.205.190:42123 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
I20260808 02:57:27.953804 6152 ts_manager.cc:194] Registered new tserver with Master: 88b7a34f13ce47c398320aa26ea4b6db (127.0.205.129:46095)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
I20260808 02:57:27.955570 6152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:58373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:57:27.986078 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:42123
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:28.335399 6444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:28.335748 6444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:28.335912 6444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:28.343348 6444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:28.343519 6444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:28.343698 6444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:28.353796 6444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:42123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:28.355345 6444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:28.356747 6444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:28.371376 6451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:28.371508 6450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:28.371428 6453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:28.372560 6444 server_base.cc:1034] running on GCE node
I20260808 02:57:28.373332 6444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:28.374589 6444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:28.375857 6444 hybrid_clock.cc:648] HybridClock initialized: now 1786157848375813 us; error 37 us; skew 500 ppm
I20260808 02:57:28.376231 6444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:28 dist-test-slave-6ntk krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:28.384130 6444 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:57:28.385756 6444 webserver.cc:460] Webserver started at http://127.0.205.130:35531/ using document root <none> and password file <none>
I20260808 02:57:28.386513 6444 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:28.386669 6444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:28.387006 6444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:28.390092 6444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "bc15685d7c924b5c8efa63446b101bcf"
format_stamp: "Formatted at 2026-08-08 02:57:28 on dist-test-slave-6ntk"
server_key: "78bbd797c7881f20592c728c31bd06a8"
server_key_iv: "7b72ddfdc1177060161a62f6c81e6a90"
server_key_version: "encryptionkey@0"
I20260808 02:57:28.390889 6444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "bc15685d7c924b5c8efa63446b101bcf"
format_stamp: "Formatted at 2026-08-08 02:57:28 on dist-test-slave-6ntk"
server_key: "78bbd797c7881f20592c728c31bd06a8"
server_key_iv: "7b72ddfdc1177060161a62f6c81e6a90"
server_key_version: "encryptionkey@0"
I20260808 02:57:28.396297 6444 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:57:28.400033 6460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:28.401625 6444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 02:57:28.401813 6444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "bc15685d7c924b5c8efa63446b101bcf"
format_stamp: "Formatted at 2026-08-08 02:57:28 on dist-test-slave-6ntk"
server_key: "78bbd797c7881f20592c728c31bd06a8"
server_key_iv: "7b72ddfdc1177060161a62f6c81e6a90"
server_key_version: "encryptionkey@0"
I20260808 02:57:28.402113 6444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:28.429128 6444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:28.433969 6444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:28.434302 6444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:28.435343 6444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:28.436745 6444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:28.436858 6444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:28.437021 6444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:28.437098 6444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:28.472407 6444 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:39775
I20260808 02:57:28.472452 6618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:39775 every 8 connection(s)
I20260808 02:57:28.473826 6444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:28.482594 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 6444
I20260808 02:57:28.482790 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:57:28.483280 822 external_mini_cluster.cc:1428] Setting key 5291fdbdeda2350a730658a61b972c82
Aug 08 02:57:28 dist-test-slave-6ntk krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:28.497462 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:28.476609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:57307 (local address 127.0.205.190:42123)
0808 02:57:28.477090 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:28.477095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:28.478034 (+ 939us) server_negotiation.cc:407] Connection header received
0808 02:57:28.479903 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:28.479908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:28.480005 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:28.480133 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:28.481768 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:28.482871 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:28.485420 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:28.486051 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:28.486216 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:28.490180 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:28.490189 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:28.490196 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:28.490243 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:28.494002 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:28.495040 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:28.495044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:28.495046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:28.495182 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:28.495707 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:28.495710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:28.495712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:28.495972 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:28.496075 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:28.497002 (+ 927us) server_negotiation.cc:299] Negotiation successful
0808 02:57:28.497169 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":267,"threads_started":1}
I20260808 02:57:28.499243 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:28.477067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:42123 (local address 127.0.205.130:57307)
0808 02:57:28.477808 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:28.477845 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:28.479619 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:28.480407 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:28.480416 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:28.480950 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:28.481524 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:28.481542 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:28.483061 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:28.483066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:28.485134 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:28.485153 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:28.486372 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:28.486379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:28.486612 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:28.487383 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:28.487408 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:28.489918 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:28.494158 (+ 4240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:28.494171 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:28.494193 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:28.494905 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:28.495316 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:28.495324 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:28.495330 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:28.495596 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:28.496118 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:28.496129 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:28.496613 (+ 484us) client_negotiation.cc:769] Sending connection context
0808 02:57:28.496948 (+ 335us) client_negotiation.cc:240] Negotiation successful
0808 02:57:28.497601 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":296,"threads_started":1}
I20260808 02:57:28.501073 6619 heartbeater.cc:347] Connected to a master server at 127.0.205.190:42123
I20260808 02:57:28.501456 6619 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:28.502351 6619 heartbeater.cc:511] Master 127.0.205.190:42123 requested a full tablet report, sending...
I20260808 02:57:28.504154 6152 ts_manager.cc:194] Registered new tserver with Master: bc15685d7c924b5c8efa63446b101bcf (127.0.205.130:39775)
I20260808 02:57:28.505504 6152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:57307
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:57:28.541034 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:42123
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:28.878687 6627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:28.879025 6627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:28.879204 6627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:28.886695 6627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:28.886863 6627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:28.887043 6627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:28.897177 6627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:42123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:28.898648 6627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:28.900017 6627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:28.914554 6635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:28.914636 6634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:28.914613 6639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:28.915656 6627 server_base.cc:1034] running on GCE node
I20260808 02:57:28.916272 6627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:28.917281 6627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:28.918483 6627 hybrid_clock.cc:648] HybridClock initialized: now 1786157848918435 us; error 35 us; skew 500 ppm
I20260808 02:57:28.918804 6627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:28 dist-test-slave-6ntk krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:28.921962 6627 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:57:28.923579 6627 webserver.cc:460] Webserver started at http://127.0.205.131:33491/ using document root <none> and password file <none>
I20260808 02:57:28.924273 6627 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:28.924423 6627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:28.924860 6627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:28.927953 6627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "043a82745f6b4676b84a64b65ffffdc4"
format_stamp: "Formatted at 2026-08-08 02:57:28 on dist-test-slave-6ntk"
server_key: "94ff9139e7c9dce37ff7d6c27bd27c02"
server_key_iv: "83c6272d6d4f06544bcf4ede650bfa98"
server_key_version: "encryptionkey@0"
I20260808 02:57:28.928752 6627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "043a82745f6b4676b84a64b65ffffdc4"
format_stamp: "Formatted at 2026-08-08 02:57:28 on dist-test-slave-6ntk"
server_key: "94ff9139e7c9dce37ff7d6c27bd27c02"
server_key_iv: "83c6272d6d4f06544bcf4ede650bfa98"
server_key_version: "encryptionkey@0"
I20260808 02:57:28.934211 6627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:57:28.937790 6648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:28.939248 6627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:28.939428 6627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "043a82745f6b4676b84a64b65ffffdc4"
format_stamp: "Formatted at 2026-08-08 02:57:28 on dist-test-slave-6ntk"
server_key: "94ff9139e7c9dce37ff7d6c27bd27c02"
server_key_iv: "83c6272d6d4f06544bcf4ede650bfa98"
server_key_version: "encryptionkey@0"
I20260808 02:57:28.939651 6627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:28.958801 6432 heartbeater.cc:503] Master 127.0.205.190:42123 was elected leader, sending a full tablet report...
I20260808 02:57:28.959612 6627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:28.964291 6627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:28.964605 6627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:28.965653 6627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:28.967114 6627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:28.967223 6627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:28.967391 6627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:28.967478 6627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:29.013515 6627 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:41095
I20260808 02:57:29.013556 6812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:41095 every 8 connection(s)
I20260808 02:57:29.014950 6627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:57:29.024911 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 6627
I20260808 02:57:29.025163 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:57:29.025637 822 external_mini_cluster.cc:1428] Setting key bed5bb13cde3f6c955ddfce851f85628
Aug 08 02:57:29 dist-test-slave-6ntk krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:29.035000 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:29.017598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:48371 (local address 127.0.205.190:42123)
0808 02:57:29.018120 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:29.018125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:29.019024 (+ 899us) server_negotiation.cc:407] Connection header received
0808 02:57:29.020852 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:29.020858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:29.020954 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:29.021082 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:29.022668 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.023450 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.025351 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.025804 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.025990 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:29.029768 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:29.029772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:29.029776 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:29.029806 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:29.032346 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.033013 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.033017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.033019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.033137 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.033504 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.033507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.033509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.033755 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:29.033863 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:29.034546 (+ 683us) server_negotiation.cc:299] Negotiation successful
0808 02:57:29.034707 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":308,"threads_started":1}
I20260808 02:57:29.036051 6817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:29.018117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:42123 (local address 127.0.205.131:48371)
0808 02:57:29.018851 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:29.018899 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:29.020571 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:29.021343 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:29.021352 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:29.021854 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:29.022476 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:29.022493 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.023616 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.023620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:29.025167 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:29.025184 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.026102 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.026109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:29.026385 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:29.027149 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:29.027194 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:29.029567 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:29.032473 (+ 2906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:29.032480 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:29.032492 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:29.032889 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:29.033230 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:29.033233 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:29.033235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:29.033410 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:29.033859 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:29.033865 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:29.034241 (+ 376us) client_negotiation.cc:769] Sending connection context
0808 02:57:29.034474 (+ 233us) client_negotiation.cc:240] Negotiation successful
0808 02:57:29.034840 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":298,"threads_started":1}
I20260808 02:57:29.037366 6813 heartbeater.cc:347] Connected to a master server at 127.0.205.190:42123
I20260808 02:57:29.037685 6813 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:29.038487 6813 heartbeater.cc:511] Master 127.0.205.190:42123 requested a full tablet report, sending...
I20260808 02:57:29.040067 6151 ts_manager.cc:194] Registered new tserver with Master: 043a82745f6b4676b84a64b65ffffdc4 (127.0.205.131:41095)
I20260808 02:57:29.041213 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:57:29.041321 6151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:48371
Aug 08 02:57:29 dist-test-slave-6ntk krb5kdc[6092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:57:29 dist-test-slave-6ntk krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:29.098135 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:29.082389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40906 (local address 127.0.205.129:46095)
0808 02:57:29.083025 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:29.083032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:29.083060 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:57:29.083121 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:29.083128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:29.083345 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:29.083511 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:29.084987 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.086123 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.087331 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.087908 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.088005 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:29.092133 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:29.092146 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:29.092162 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:29.092201 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:29.095292 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.096031 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.096040 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.096043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.096188 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.096541 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.096544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.096546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.096936 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:29.097194 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:29.097444 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 02:57:29.097657 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":311,"threads_started":1}
Aug 08 02:57:29 dist-test-slave-6ntk krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:29.118780 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:29.103461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35264 (local address 127.0.205.190:42123)
0808 02:57:29.103704 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:29.103709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:29.104055 (+ 346us) server_negotiation.cc:407] Connection header received
0808 02:57:29.104243 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:29.104248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:29.104319 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:29.104410 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:29.105723 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.106825 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.108587 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.108997 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.109097 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:29.113061 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:29.113066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:29.113070 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:29.113108 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:29.115727 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.116562 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.116567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.116569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.116766 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.117216 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.117222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.117225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.117598 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:29.117856 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:29.118207 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 02:57:29.118449 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
W20260808 02:57:29.119361 6151 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35264
I20260808 02:57:29.120451 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 6252
I20260808 02:57:29.133961 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:29.121365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40908 (local address 127.0.205.129:46095)
0808 02:57:29.121616 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:29.121622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:29.121834 (+ 212us) server_negotiation.cc:407] Connection header received
0808 02:57:29.122081 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:29.122086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:29.122186 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:29.122296 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:29.123530 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.124283 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.125598 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.126095 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.126239 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:29.128611 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:29.128617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:29.128620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:29.128654 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:29.131232 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.131898 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.131907 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.131911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.132054 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.132453 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.132459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.132462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.132733 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:29.132994 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:29.133321 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 02:57:29.133577 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 02:57:29.134582 6427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:29.221535 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 6252
I20260808 02:57:29.247581 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 6444
Aug 08 02:57:29 dist-test-slave-6ntk krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:57:29.268450 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:29.248645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44698 (local address 127.0.205.130:39775)
0808 02:57:29.249287 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:29.249294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:29.249323 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:57:29.249385 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:29.249391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:29.249612 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:29.249765 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:29.251326 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.252366 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.253765 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.254874 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.255056 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:29.259859 (+ 4803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:29.259879 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:29.259902 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:29.259965 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:29.264518 (+ 4553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.265503 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.265551 (+ 48us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.265556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.265797 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.266412 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.266420 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.266426 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.266939 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:29.267333 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:29.267708 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 02:57:29.268021 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":335,"threads_started":1}
I20260808 02:57:29.269838 6612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:29.356297 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 6444
I20260808 02:57:29.378636 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 6627
Aug 08 02:57:29 dist-test-slave-6ntk krb5kdc[6092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:57:29.395853 6839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:29.379777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55030 (local address 127.0.205.131:41095)
0808 02:57:29.380430 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:29.380437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:29.380468 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:57:29.380531 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:29.380537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:29.380747 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:29.380882 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:29.382224 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.383329 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.384640 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:29.385205 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:29.385303 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:29.389642 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:29.389654 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:29.389669 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:29.389713 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:29.392806 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.393603 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.393610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.393612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.393746 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:29.394284 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:29.394288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:29.394290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:29.394660 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:29.394930 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:29.395219 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 02:57:29.395437 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":344,"threads_started":1}
I20260808 02:57:29.397300 6807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:29.482025 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 6627
I20260808 02:57:29.508087 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 6112
I20260808 02:57:29.509403 6198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:29.617914 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 6112
2026-08-08T02:57:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4386 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:57:29 dist-test-slave-6ntk krb5kdc[6848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:57:29 dist-test-slave-6ntk krb5kdc[6848](info): set up 2 sockets
Aug 08 02:57:29 dist-test-slave-6ntk krb5kdc[6848](info): commencing operation
krb5kdc: starting...
W20260808 02:57:30.946043 822 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:57:31 dist-test-slave-6ntk krb5kdc[6848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:57:31.185324 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:46403
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:46403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:31.557060 6867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:31.557405 6867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:31.557583 6867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:31.565089 6867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:31.565231 6867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:31.565352 6867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:31.565435 6867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:31.565521 6867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:31.575965 6867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:46403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:46403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:31.577461 6867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:31.578845 6867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:31.590734 6876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:31.591032 6873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:31.591554 6872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:31.592320 6867 server_base.cc:1034] running on GCE node
I20260808 02:57:31.593137 6867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:31.594621 6867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:31.595845 6867 hybrid_clock.cc:648] HybridClock initialized: now 1786157851595793 us; error 39 us; skew 500 ppm
I20260808 02:57:31.596293 6867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:31 dist-test-slave-6ntk krb5kdc[6848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:31.599560 6867 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:57:31.600677 6867 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:31.600767 6867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:31.601030 6867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:31.604274 6867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "173e1a490a9e4f4e8962027bd525d3d3"
format_stamp: "Formatted at 2026-08-08 02:57:31 on dist-test-slave-6ntk"
server_key: "6f3949747f31599ab2845529616e237a"
server_key_iv: "237676941d15f9dcafa6fe868cb7ac8f"
server_key_version: "encryptionkey@0"
I20260808 02:57:31.605093 6867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "173e1a490a9e4f4e8962027bd525d3d3"
format_stamp: "Formatted at 2026-08-08 02:57:31 on dist-test-slave-6ntk"
server_key: "6f3949747f31599ab2845529616e237a"
server_key_iv: "237676941d15f9dcafa6fe868cb7ac8f"
server_key_version: "encryptionkey@0"
I20260808 02:57:31.610538 6867 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 02:57:31.614240 6882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:31.615767 6867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:57:31.615985 6867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "173e1a490a9e4f4e8962027bd525d3d3"
format_stamp: "Formatted at 2026-08-08 02:57:31 on dist-test-slave-6ntk"
server_key: "6f3949747f31599ab2845529616e237a"
server_key_iv: "237676941d15f9dcafa6fe868cb7ac8f"
server_key_version: "encryptionkey@0"
I20260808 02:57:31.616240 6867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:31.632617 6867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:31.637570 6867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:31.637816 6867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:31.655571 6867 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:46403
I20260808 02:57:31.655596 6953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:46403 every 8 connection(s)
I20260808 02:57:31.656553 6867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:31.660396 6954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:31.660452 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 6867
I20260808 02:57:31.660733 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:31.661449 822 external_mini_cluster.cc:1428] Setting key 4513635e551b73b098ae7f034b440950
I20260808 02:57:31.675611 6954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3: Bootstrap starting.
Aug 08 02:57:31 dist-test-slave-6ntk krb5kdc[6848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:31.679033 6954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:31.680388 6954 log.cc:824] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:31.683645 6954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3: No bootstrap required, opened a new log
I20260808 02:57:31.687698 6960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:31.663623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46358 (local address 127.0.205.190:46403)
0808 02:57:31.664520 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:31.664535 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:31.664592 (+ 57us) server_negotiation.cc:407] Connection header received
0808 02:57:31.667485 (+ 2893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:31.667518 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:31.668077 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:31.668602 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:31.670523 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:31.672631 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:31.674044 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:31.675088 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:31.675283 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:31.679609 (+ 4326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:31.679620 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:31.679639 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:31.679689 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:31.683272 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:31.684182 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:31.684192 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:31.684198 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:31.684365 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:31.684825 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:31.684829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:31.684831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:31.685377 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:31.685650 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:31.686203 (+ 553us) server_negotiation.cc:299] Negotiation successful
0808 02:57:31.686630 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":363,"threads_started":1}
I20260808 02:57:31.692301 6954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "173e1a490a9e4f4e8962027bd525d3d3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46403 } }
I20260808 02:57:31.692677 6954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:31.692804 6954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173e1a490a9e4f4e8962027bd525d3d3, State: Initialized, Role: FOLLOWER
I20260808 02:57:31.693418 6954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "173e1a490a9e4f4e8962027bd525d3d3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46403 } }
I20260808 02:57:31.693661 6954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:31.693800 6954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:31.693996 6954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:31.696326 6954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "173e1a490a9e4f4e8962027bd525d3d3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46403 } }
I20260808 02:57:31.696869 6954 leader_election.cc:302] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 173e1a490a9e4f4e8962027bd525d3d3; no voters:
I20260808 02:57:31.697543 6954 leader_election.cc:288] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:31.697700 6962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:31.698448 6962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [term 1 LEADER]: Becoming Leader. State: Replica: 173e1a490a9e4f4e8962027bd525d3d3, State: Running, Role: LEADER
I20260808 02:57:31.698987 6962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "173e1a490a9e4f4e8962027bd525d3d3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46403 } }
I20260808 02:57:31.699708 6954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:31.702124 6963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "173e1a490a9e4f4e8962027bd525d3d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "173e1a490a9e4f4e8962027bd525d3d3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46403 } } }
I20260808 02:57:31.702524 6963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:31.703431 6965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 173e1a490a9e4f4e8962027bd525d3d3. Latest consensus state: current_term: 1 leader_uuid: "173e1a490a9e4f4e8962027bd525d3d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "173e1a490a9e4f4e8962027bd525d3d3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46403 } } }
I20260808 02:57:31.706590 6965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:31.707315 6972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:31.712987 6972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:31.722044 6972 catalog_manager.cc:1347] Generated new cluster ID: c2f2ce2949224e648e2bb1ef4fddfbc5
I20260808 02:57:31.722172 6972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:31.740953 6972 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:31.742367 6972 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:31.749830 6972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3: Generated new TSK 0
I20260808 02:57:31.750721 6972 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:57:31.811399 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46403
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:32.170709 6992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:32.170995 6992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:32.171188 6992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:32.178453 6992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:32.178560 6992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:32.178769 6992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:32.191619 6992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:32.193244 6992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:32.194710 6992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:32.208213 7002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:32.208184 7004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:32.208253 7000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:32.209451 6992 server_base.cc:1034] running on GCE node
I20260808 02:57:32.210323 6992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:32.211398 6992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:32.212612 6992 hybrid_clock.cc:648] HybridClock initialized: now 1786157852212564 us; error 56 us; skew 500 ppm
I20260808 02:57:32.212968 6992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:32 dist-test-slave-6ntk krb5kdc[6848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157852, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:32.216428 6992 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:57:32.217721 6992 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:32.217921 6992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:32.218289 6992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:32.221861 6992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "c0e8622914c44eba8bf269b464d7dd07"
format_stamp: "Formatted at 2026-08-08 02:57:32 on dist-test-slave-6ntk"
server_key: "985f2edefe13fbd4be6ed5dd74f59213"
server_key_iv: "e0da4b8ceece44be139213817a0385b5"
server_key_version: "encryptionkey@0"
I20260808 02:57:32.222822 6992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "c0e8622914c44eba8bf269b464d7dd07"
format_stamp: "Formatted at 2026-08-08 02:57:32 on dist-test-slave-6ntk"
server_key: "985f2edefe13fbd4be6ed5dd74f59213"
server_key_iv: "e0da4b8ceece44be139213817a0385b5"
server_key_version: "encryptionkey@0"
I20260808 02:57:32.228191 6992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:57:32.231946 7012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:32.233456 6992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:57:32.233651 6992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "c0e8622914c44eba8bf269b464d7dd07"
format_stamp: "Formatted at 2026-08-08 02:57:32 on dist-test-slave-6ntk"
server_key: "985f2edefe13fbd4be6ed5dd74f59213"
server_key_iv: "e0da4b8ceece44be139213817a0385b5"
server_key_version: "encryptionkey@0"
I20260808 02:57:32.233986 6992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:32.256511 6992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:32.261624 6992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:32.262023 6992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:32.263130 6992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:32.264611 6992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:32.264747 6992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:32.264894 6992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:32.264991 6992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:32.299991 6992 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:38459
I20260808 02:57:32.300043 7175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:38459 every 8 connection(s)
I20260808 02:57:32.300984 6992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:32.307243 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 6992
I20260808 02:57:32.307500 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:57:32.308338 822 external_mini_cluster.cc:1428] Setting key b27504f4d439d1fe9444fff75edfb839
Aug 08 02:57:32 dist-test-slave-6ntk krb5kdc[6848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157852, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:32.327488 7178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:32.303593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:53645 (local address 127.0.205.190:46403)
0808 02:57:32.304067 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:32.304071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:32.304916 (+ 845us) server_negotiation.cc:407] Connection header received
0808 02:57:32.307928 (+ 3012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:32.307934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:32.308041 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:32.308163 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:32.310256 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:32.311030 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:32.313839 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:32.314485 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:32.314617 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:32.319531 (+ 4914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:32.319539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:32.319546 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:32.319599 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:32.323658 (+ 4059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:32.324551 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:32.324559 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:32.324566 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:32.324729 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:32.325186 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:32.325194 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:32.325200 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:32.325640 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:32.325833 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:32.326695 (+ 862us) server_negotiation.cc:299] Negotiation successful
0808 02:57:32.326965 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":254,"threads_started":1}
I20260808 02:57:32.328187 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:32.304031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:46403 (local address 127.0.205.129:53645)
0808 02:57:32.304745 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:32.304793 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:32.307570 (+ 2777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:32.308486 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:32.308499 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:32.309227 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:32.310052 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:32.310078 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:32.311292 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:32.311301 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:32.313596 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:32.313614 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:32.314740 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:32.314750 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:32.315092 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:32.315948 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:32.315986 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:32.319234 (+ 3248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:32.323847 (+ 4613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:32.323856 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:32.323871 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:32.324372 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:32.324850 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:32.324853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:32.324856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:32.325058 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:32.325792 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:32.325799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:32.326252 (+ 453us) client_negotiation.cc:769] Sending connection context
0808 02:57:32.326514 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 02:57:32.326957 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":293,"threads_started":1}
I20260808 02:57:32.329942 7176 heartbeater.cc:347] Connected to a master server at 127.0.205.190:46403
I20260808 02:57:32.330320 7176 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:32.331036 7176 heartbeater.cc:511] Master 127.0.205.190:46403 requested a full tablet report, sending...
I20260808 02:57:32.333403 6906 ts_manager.cc:194] Registered new tserver with Master: c0e8622914c44eba8bf269b464d7dd07 (127.0.205.129:38459)
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
I20260808 02:57:32.335142 6906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:53645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:57:32.367147 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46403
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:32.715117 7185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:32.715425 7185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:32.715525 7185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:32.723307 7185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:32.723440 7185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:32.723569 7185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:32.733772 7185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:32.735407 7185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:32.736732 7185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:32.750599 7198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:32.750622 7194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:32.750808 7192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:32.751562 7185 server_base.cc:1034] running on GCE node
I20260808 02:57:32.752244 7185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:32.753258 7185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:32.754499 7185 hybrid_clock.cc:648] HybridClock initialized: now 1786157852754429 us; error 63 us; skew 500 ppm
I20260808 02:57:32.754845 7185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:32 dist-test-slave-6ntk krb5kdc[6848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157852, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:32.758304 7185 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:57:32.759526 7185 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:32.759671 7185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:32.760023 7185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:32.763185 7185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "eb3901b865a64ede9d640e54857d6712"
format_stamp: "Formatted at 2026-08-08 02:57:32 on dist-test-slave-6ntk"
server_key: "c469d595f879156bfcb250627f2ee4bc"
server_key_iv: "044d8f09244538fb04ed19bf296a2b93"
server_key_version: "encryptionkey@0"
I20260808 02:57:32.764037 7185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "eb3901b865a64ede9d640e54857d6712"
format_stamp: "Formatted at 2026-08-08 02:57:32 on dist-test-slave-6ntk"
server_key: "c469d595f879156bfcb250627f2ee4bc"
server_key_iv: "044d8f09244538fb04ed19bf296a2b93"
server_key_version: "encryptionkey@0"
I20260808 02:57:32.769382 7185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 02:57:32.772979 7205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:32.774420 7185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:32.774606 7185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "eb3901b865a64ede9d640e54857d6712"
format_stamp: "Formatted at 2026-08-08 02:57:32 on dist-test-slave-6ntk"
server_key: "c469d595f879156bfcb250627f2ee4bc"
server_key_iv: "044d8f09244538fb04ed19bf296a2b93"
server_key_version: "encryptionkey@0"
I20260808 02:57:32.774844 7185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:32.791960 7185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:32.796087 7185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:32.796380 7185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:32.797374 7185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:32.798755 7185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:32.798880 7185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:32.799031 7185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:32.799119 7185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:32.833418 7185 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:35329
I20260808 02:57:32.833474 7355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:35329 every 8 connection(s)
I20260808 02:57:32.834406 7185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:32.841274 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 7185
I20260808 02:57:32.841508 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:57:32.842139 822 external_mini_cluster.cc:1428] Setting key ee43ffbfd2533f41d6987a485504ce96
Aug 08 02:57:32 dist-test-slave-6ntk krb5kdc[6848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157852, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:32.858283 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:32.836966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:41069 (local address 127.0.205.190:46403)
0808 02:57:32.837448 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:32.837453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:32.838396 (+ 943us) server_negotiation.cc:407] Connection header received
0808 02:57:32.840338 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:32.840345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:32.840444 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:32.840569 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:32.842781 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:32.843671 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:32.846047 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:32.846498 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:32.846598 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:32.851083 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:32.851088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:32.851092 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:32.851127 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:32.854530 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:32.855371 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:32.855380 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:32.855385 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:32.855559 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:32.856087 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:32.856094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:32.856100 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:32.856509 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:32.856669 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:32.857447 (+ 778us) server_negotiation.cc:299] Negotiation successful
0808 02:57:32.857738 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":265,"threads_started":1}
I20260808 02:57:32.859076 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:32.837457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:46403 (local address 127.0.205.130:41069)
0808 02:57:32.838230 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:32.838269 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:32.840045 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:32.840805 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:32.840813 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:32.841663 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:32.842545 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:32.842579 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:32.843850 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:32.843855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:32.845842 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:32.845862 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:32.846745 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:32.846755 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:32.847074 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:32.847884 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:32.847920 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:32.850798 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:32.854690 (+ 3892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:32.854699 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:32.854715 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:32.855212 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:32.855717 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:32.855721 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:32.855724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:32.855950 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:32.856652 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:32.856658 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:32.857035 (+ 377us) client_negotiation.cc:769] Sending connection context
0808 02:57:32.857284 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 02:57:32.857741 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":364,"threads_started":1}
I20260808 02:57:32.860800 7356 heartbeater.cc:347] Connected to a master server at 127.0.205.190:46403
I20260808 02:57:32.861173 7356 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:32.862025 7356 heartbeater.cc:511] Master 127.0.205.190:46403 requested a full tablet report, sending...
I20260808 02:57:32.863824 6906 ts_manager.cc:194] Registered new tserver with Master: eb3901b865a64ede9d640e54857d6712 (127.0.205.130:35329)
I20260808 02:57:32.865206 6906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:41069
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:57:32.898466 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46403
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:33.243819 7365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:33.244122 7365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:33.244225 7365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:33.251451 7365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:33.251572 7365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:33.251698 7365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:33.261551 7365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:33.263162 7365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:33.264464 7365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:33.278949 7370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:33.278954 7372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:33.278992 7376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:33.279853 7365 server_base.cc:1034] running on GCE node
I20260808 02:57:33.280586 7365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:33.281663 7365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:33.282939 7365 hybrid_clock.cc:648] HybridClock initialized: now 1786157853282856 us; error 56 us; skew 500 ppm
I20260808 02:57:33.283300 7365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:33 dist-test-slave-6ntk krb5kdc[6848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157853, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:33.286784 7365 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:57:33.288033 7365 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:33.288173 7365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:33.288595 7365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:33.291759 7365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "9012b175d7cf4087a3e8246fde62b153"
format_stamp: "Formatted at 2026-08-08 02:57:33 on dist-test-slave-6ntk"
server_key: "573a06bd0cfec83f14a32eab888f0ce9"
server_key_iv: "1fa2bce80ead34f857f61cc8a481627d"
server_key_version: "encryptionkey@0"
I20260808 02:57:33.292636 7365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "9012b175d7cf4087a3e8246fde62b153"
format_stamp: "Formatted at 2026-08-08 02:57:33 on dist-test-slave-6ntk"
server_key: "573a06bd0cfec83f14a32eab888f0ce9"
server_key_iv: "1fa2bce80ead34f857f61cc8a481627d"
server_key_version: "encryptionkey@0"
I20260808 02:57:33.298002 7365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 02:57:33.302184 7383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:33.303941 7365 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:57:33.304136 7365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "9012b175d7cf4087a3e8246fde62b153"
format_stamp: "Formatted at 2026-08-08 02:57:33 on dist-test-slave-6ntk"
server_key: "573a06bd0cfec83f14a32eab888f0ce9"
server_key_iv: "1fa2bce80ead34f857f61cc8a481627d"
server_key_version: "encryptionkey@0"
I20260808 02:57:33.304397 7365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:33.324306 7365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:33.328583 7365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:33.328894 7365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:33.330190 7365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:33.331681 7365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:33.331830 7365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:33.331982 7365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:33.332091 7365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:33.338809 7176 heartbeater.cc:503] Master 127.0.205.190:46403 was elected leader, sending a full tablet report...
I20260808 02:57:33.368561 7365 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:44811
I20260808 02:57:33.368664 7546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:44811 every 8 connection(s)
I20260808 02:57:33.369483 7365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:57:33.372913 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 7365
I20260808 02:57:33.373164 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:57:33.373756 822 external_mini_cluster.cc:1428] Setting key 7d102c9726d4e2153e890481a2a526c3
Aug 08 02:57:33 dist-test-slave-6ntk krb5kdc[6848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157853, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:33.397336 7551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.372132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:34819 (local address 127.0.205.190:46403)
0808 02:57:33.372660 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:33.372666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:33.373719 (+ 1053us) server_negotiation.cc:407] Connection header received
0808 02:57:33.376295 (+ 2576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:33.376301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:33.376410 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:33.376533 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:33.378270 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.379132 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.380711 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.381141 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.381237 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.391143 (+ 9906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:33.391150 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:33.391156 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:33.391200 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:33.393782 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:33.394706 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:33.394712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:33.394716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:33.394861 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:33.395367 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:33.395378 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:33.395380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:33.395673 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:33.395856 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:33.396635 (+ 779us) server_negotiation.cc:299] Negotiation successful
0808 02:57:33.396933 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":243,"threads_started":1}
I20260808 02:57:33.398473 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.372653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:46403 (local address 127.0.205.131:34819)
0808 02:57:33.373522 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:33.373572 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:33.375942 (+ 2370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:33.376722 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:33.376730 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:33.377470 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:33.378100 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.378119 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.379270 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.379274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.380575 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.380587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.381326 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.381330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.381513 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.387873 (+ 6360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:33.387910 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:33.390903 (+ 2993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:33.394026 (+ 3123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:33.394037 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:33.394057 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:33.394561 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:33.394983 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:33.394989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:33.394992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:33.395201 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:33.395826 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:33.395835 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:33.396221 (+ 386us) client_negotiation.cc:769] Sending connection context
0808 02:57:33.396518 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 02:57:33.397130 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":334,"threads_started":1}
I20260808 02:57:33.400236 7548 heartbeater.cc:347] Connected to a master server at 127.0.205.190:46403
I20260808 02:57:33.400650 7548 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:33.401592 7548 heartbeater.cc:511] Master 127.0.205.190:46403 requested a full tablet report, sending...
I20260808 02:57:33.403301 6906 ts_manager.cc:194] Registered new tserver with Master: 9012b175d7cf4087a3e8246fde62b153 (127.0.205.131:44811)
I20260808 02:57:33.404135 6906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:34819
I20260808 02:57:33.415108 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:57:33.433099 7551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.419908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46370 (local address 127.0.205.190:46403)
0808 02:57:33.420099 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:33.420104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:33.420707 (+ 603us) server_negotiation.cc:407] Connection header received
0808 02:57:33.420877 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:33.420882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:33.421006 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:33.421132 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:33.422703 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.423507 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.424763 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.425233 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.425354 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.428009 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:33.428014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:33.428018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:33.428051 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:33.430542 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:33.431198 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:33.431202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:33.431204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:33.431338 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:33.431694 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:33.431697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:33.431699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:33.432068 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:33.432250 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:33.432640 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 02:57:33.432831 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:57:33.438076 6906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:57:33.444921 6906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:33.487991 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.457722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33630 (local address 127.0.205.130:35329)
0808 02:57:33.479105 (+ 21383us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:33.479115 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:33.479148 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:57:33.479284 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:33.479292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:33.479536 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:33.480717 (+ 1181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:33.480989 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.482585 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.485255 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.486721 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.486869 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.487081 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:33.487460 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 02:57:33.487649 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21198,"thread_start_us":21010,"threads_started":1}
I20260808 02:57:33.488078 7565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.457436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:35329 (local address 127.0.0.1:33630)
0808 02:57:33.473341 (+ 15905us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:33.473406 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:33.473696 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:33.479850 (+ 6154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:33.479859 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:33.479897 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:33.480496 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.480510 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.482732 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.482738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.485083 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.485098 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.486931 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.486938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.487145 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.487199 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 02:57:33.487366 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 02:57:33.487576 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15494,"mutex_wait_us":12,"thread_start_us":15290,"threads_started":1}
I20260808 02:57:33.488459 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.459104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42224 (local address 127.0.205.129:38459)
0808 02:57:33.479523 (+ 20419us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:33.479531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:33.480302 (+ 771us) server_negotiation.cc:407] Connection header received
0808 02:57:33.480417 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:33.480426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:33.480646 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:33.480830 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:33.481729 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.483086 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.485107 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.486721 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.486869 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.487117 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:33.487872 (+ 755us) server_negotiation.cc:299] Negotiation successful
0808 02:57:33.488070 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20252,"thread_start_us":20065,"threads_started":1}
I20260808 02:57:33.489120 7568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.458412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:38459 (local address 127.0.0.1:42224)
0808 02:57:33.479982 (+ 21570us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:33.480006 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:33.480160 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:33.481076 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:33.481081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:33.481108 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:33.481519 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.481533 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.483228 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.483235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.484934 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.484947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.487264 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.487269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.487453 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.487486 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:57:33.487575 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 02:57:33.487693 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21425,"mutex_wait_us":131,"thread_start_us":21271,"threads_started":1}
I20260808 02:57:33.489609 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.458130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:44811 (local address 127.0.0.1:42752)
0808 02:57:33.478565 (+ 20435us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:33.478591 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:33.478748 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:33.479949 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:33.479953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:33.479994 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:33.480498 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.480509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.482383 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.482390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.485540 (+ 3150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.485553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.487867 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.487873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.488509 (+ 636us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.488676 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 02:57:33.488789 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 02:57:33.488939 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20265,"mutex_wait_us":5,"thread_start_us":20196,"threads_started":1}
I20260808 02:57:33.489794 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.458899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.0.205.131:44811)
0808 02:57:33.477807 (+ 18908us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:33.477820 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:33.478903 (+ 1083us) server_negotiation.cc:407] Connection header received
0808 02:57:33.479044 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:33.479056 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:33.479364 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:33.479618 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:33.480687 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.482237 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.486168 (+ 3931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.487384 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.487497 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.487707 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:33.489224 (+ 1517us) server_negotiation.cc:299] Negotiation successful
0808 02:57:33.489392 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18662,"thread_start_us":18934,"threads_started":1}
I20260808 02:57:33.492259 7269 tablet_service.cc:1467] Processing CreateTablet for tablet 5e577e29b09e49119e059178ef872428 (DEFAULT_TABLE table=test-table [id=1a6b2b8f94394278af6712d265b59eaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:33.493554 7269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e577e29b09e49119e059178ef872428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:33.493038 7090 tablet_service.cc:1467] Processing CreateTablet for tablet 5e577e29b09e49119e059178ef872428 (DEFAULT_TABLE table=test-table [id=1a6b2b8f94394278af6712d265b59eaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:33.494550 7090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e577e29b09e49119e059178ef872428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:33.493988 7456 tablet_service.cc:1467] Processing CreateTablet for tablet 5e577e29b09e49119e059178ef872428 (DEFAULT_TABLE table=test-table [id=1a6b2b8f94394278af6712d265b59eaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:33.495419 7456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e577e29b09e49119e059178ef872428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:33.505628 7572 tablet_bootstrap.cc:492] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: Bootstrap starting.
I20260808 02:57:33.509522 7572 tablet_bootstrap.cc:654] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:33.510619 7574 tablet_bootstrap.cc:492] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: Bootstrap starting.
I20260808 02:57:33.510946 7572 log.cc:824] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:33.514163 7575 tablet_bootstrap.cc:492] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: Bootstrap starting.
I20260808 02:57:33.514700 7574 tablet_bootstrap.cc:654] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:33.514786 7572 tablet_bootstrap.cc:492] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: No bootstrap required, opened a new log
I20260808 02:57:33.515084 7572 ts_tablet_manager.cc:1397] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260808 02:57:33.516044 7574 log.cc:824] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:33.517467 7575 tablet_bootstrap.cc:654] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:33.518672 7575 log.cc:824] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:33.519526 7574 tablet_bootstrap.cc:492] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: No bootstrap required, opened a new log
I20260808 02:57:33.519846 7574 ts_tablet_manager.cc:1397] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 02:57:33.521756 7575 tablet_bootstrap.cc:492] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: No bootstrap required, opened a new log
I20260808 02:57:33.522202 7575 ts_tablet_manager.cc:1397] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.005s
I20260808 02:57:33.525944 7572 raft_consensus.cc:348] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 } } peers { permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 } } peers { permanent_uuid: "c0e8622914c44eba8bf269b464d7dd07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38459 } }
I20260808 02:57:33.526432 7572 raft_consensus.cc:374] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:33.526540 7572 raft_consensus.cc:729] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb3901b865a64ede9d640e54857d6712, State: Initialized, Role: FOLLOWER
I20260808 02:57:33.527208 7572 consensus_queue.cc:260] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 } } peers { permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 } } peers { permanent_uuid: "c0e8622914c44eba8bf269b464d7dd07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38459 } }
I20260808 02:57:33.528435 7356 heartbeater.cc:503] Master 127.0.205.190:46403 was elected leader, sending a full tablet report...
I20260808 02:57:33.529119 7572 ts_tablet_manager.cc:1428] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260808 02:57:33.530519 7574 raft_consensus.cc:348] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 } } peers { permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 } } peers { permanent_uuid: "c0e8622914c44eba8bf269b464d7dd07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38459 } }
I20260808 02:57:33.530966 7574 raft_consensus.cc:374] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:33.531082 7574 raft_consensus.cc:729] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e8622914c44eba8bf269b464d7dd07, State: Initialized, Role: FOLLOWER
I20260808 02:57:33.531237 7575 raft_consensus.cc:348] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 } } peers { permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 } } peers { permanent_uuid: "c0e8622914c44eba8bf269b464d7dd07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38459 } }
I20260808 02:57:33.531672 7575 raft_consensus.cc:374] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:33.531796 7575 raft_consensus.cc:729] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9012b175d7cf4087a3e8246fde62b153, State: Initialized, Role: FOLLOWER
I20260808 02:57:33.531736 7574 consensus_queue.cc:260] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 } } peers { permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 } } peers { permanent_uuid: "c0e8622914c44eba8bf269b464d7dd07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38459 } }
I20260808 02:57:33.532418 7575 consensus_queue.cc:260] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 } } peers { permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 } } peers { permanent_uuid: "c0e8622914c44eba8bf269b464d7dd07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38459 } }
I20260808 02:57:33.533617 7574 ts_tablet_manager.cc:1428] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260808 02:57:33.535079 7575 ts_tablet_manager.cc:1428] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260808 02:57:33.535547 7548 heartbeater.cc:503] Master 127.0.205.190:46403 was elected leader, sending a full tablet report...
W20260808 02:57:33.553622 7177 tablet.cc:2369] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:57:33.586616 7357 tablet.cc:2369] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:57:33.621243 7549 tablet.cc:2369] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:57:33.633567 7580 raft_consensus.cc:482] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:33.633975 7580 raft_consensus.cc:504] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 } } peers { permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 } } peers { permanent_uuid: "c0e8622914c44eba8bf269b464d7dd07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38459 } }
I20260808 02:57:33.635787 7580 leader_election.cc:288] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb3901b865a64ede9d640e54857d6712 (127.0.205.130:35329), 9012b175d7cf4087a3e8246fde62b153 (127.0.205.131:44811)
I20260808 02:57:33.643478 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.636272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:36477 (local address 127.0.205.131:44811)
0808 02:57:33.636508 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:33.636515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:33.637971 (+ 1456us) server_negotiation.cc:407] Connection header received
0808 02:57:33.638157 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:33.638164 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:33.638252 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:33.638361 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:33.639036 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.639820 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.641653 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.642527 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.642708 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.642786 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:33.643115 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 02:57:33.643242 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 02:57:33.643496 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.636289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:48139 (local address 127.0.205.130:35329)
0808 02:57:33.636508 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:33.636517 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:33.637451 (+ 934us) server_negotiation.cc:407] Connection header received
0808 02:57:33.637706 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:33.637712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:33.637795 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:33.637950 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:33.638709 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.639564 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.641553 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.642449 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.642581 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.642692 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:33.643067 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 02:57:33.643208 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:57:33.643618 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.636280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:44811 (local address 127.0.205.129:36477)
0808 02:57:33.637819 (+ 1539us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:33.637839 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:33.638006 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:33.638382 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:33.638387 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:33.638456 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:33.638844 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.638855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.639970 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.639974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.641478 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.641496 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.642786 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.642792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.642925 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.642947 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:57:33.643067 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 02:57:33.643206 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1423,"mutex_wait_us":110,"thread_start_us":1117,"threads_started":1}
I20260808 02:57:33.643987 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.636273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:35329 (local address 127.0.205.129:48139)
0808 02:57:33.637276 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:33.637303 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:33.637530 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:33.638006 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:33.638013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:33.638055 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:33.638552 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.638567 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.639703 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.639709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.641372 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:33.641386 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.642675 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.642681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:33.642888 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.642917 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:57:33.642997 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:57:33.643157 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":4,"thread_start_us":532,"threads_started":1}
I20260808 02:57:33.644804 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e577e29b09e49119e059178ef872428" candidate_uuid: "c0e8622914c44eba8bf269b464d7dd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9012b175d7cf4087a3e8246fde62b153" is_pre_election: true
I20260808 02:57:33.644997 7300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e577e29b09e49119e059178ef872428" candidate_uuid: "c0e8622914c44eba8bf269b464d7dd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb3901b865a64ede9d640e54857d6712" is_pre_election: true
I20260808 02:57:33.645411 7487 raft_consensus.cc:2454] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0e8622914c44eba8bf269b464d7dd07 in term 0.
I20260808 02:57:33.645536 7300 raft_consensus.cc:2454] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0e8622914c44eba8bf269b464d7dd07 in term 0.
I20260808 02:57:33.646263 7018 leader_election.cc:302] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9012b175d7cf4087a3e8246fde62b153, c0e8622914c44eba8bf269b464d7dd07; no voters:
I20260808 02:57:33.646725 7580 raft_consensus.cc:2781] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:33.646909 7580 raft_consensus.cc:482] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:33.647032 7580 raft_consensus.cc:3037] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:33.649284 7580 raft_consensus.cc:504] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 } } peers { permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 } } peers { permanent_uuid: "c0e8622914c44eba8bf269b464d7dd07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38459 } }
I20260808 02:57:33.649940 7580 leader_election.cc:288] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [CANDIDATE]: Term 1 election: Requested vote from peers eb3901b865a64ede9d640e54857d6712 (127.0.205.130:35329), 9012b175d7cf4087a3e8246fde62b153 (127.0.205.131:44811)
I20260808 02:57:33.650457 7300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e577e29b09e49119e059178ef872428" candidate_uuid: "c0e8622914c44eba8bf269b464d7dd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb3901b865a64ede9d640e54857d6712"
I20260808 02:57:33.650477 7487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e577e29b09e49119e059178ef872428" candidate_uuid: "c0e8622914c44eba8bf269b464d7dd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9012b175d7cf4087a3e8246fde62b153"
I20260808 02:57:33.650707 7487 raft_consensus.cc:3037] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:33.650669 7300 raft_consensus.cc:3037] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:33.653000 7487 raft_consensus.cc:2454] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e8622914c44eba8bf269b464d7dd07 in term 1.
I20260808 02:57:33.653028 7300 raft_consensus.cc:2454] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e8622914c44eba8bf269b464d7dd07 in term 1.
I20260808 02:57:33.653481 7018 leader_election.cc:302] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9012b175d7cf4087a3e8246fde62b153, c0e8622914c44eba8bf269b464d7dd07, eb3901b865a64ede9d640e54857d6712; no voters:
I20260808 02:57:33.653859 7580 raft_consensus.cc:2781] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:33.654471 7580 raft_consensus.cc:686] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 1 LEADER]: Becoming Leader. State: Replica: c0e8622914c44eba8bf269b464d7dd07, State: Running, Role: LEADER
I20260808 02:57:33.654996 7580 consensus_queue.cc:237] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 } } peers { permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 } } peers { permanent_uuid: "c0e8622914c44eba8bf269b464d7dd07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38459 } }
I20260808 02:57:33.660478 6905 catalog_manager.cc:5495] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0e8622914c44eba8bf269b464d7dd07 (127.0.205.129). New cstate: current_term: 1 leader_uuid: "c0e8622914c44eba8bf269b464d7dd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e8622914c44eba8bf269b464d7dd07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38459 } health_report { overall_health: HEALTHY } } }
I20260808 02:57:33.679507 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.673764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.0.205.129:38459)
0808 02:57:33.674031 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:33.674038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:33.674150 (+ 112us) server_negotiation.cc:407] Connection header received
0808 02:57:33.674467 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:33.674471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:33.674550 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:33.674749 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:33.675133 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.675869 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.677217 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.677622 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.677768 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.678303 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:33.678904 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:33.679063 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:33.679124 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 02:57:33.679206 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 02:57:33.688382 7300 raft_consensus.cc:1261] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712 [term 1 FOLLOWER]: Refusing update from remote peer c0e8622914c44eba8bf269b464d7dd07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:33.688382 7487 raft_consensus.cc:1261] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153 [term 1 FOLLOWER]: Refusing update from remote peer c0e8622914c44eba8bf269b464d7dd07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:33.689607 7585 consensus_queue.cc:1035] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9012b175d7cf4087a3e8246fde62b153" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 44811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:33.689997 7580 consensus_queue.cc:1035] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb3901b865a64ede9d640e54857d6712" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 35329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:33.701684 7589 mvcc.cc:205] Tried to move back new op lower bound from 7316102568695107584 to 7316102568575041536. Current Snapshot: MvccSnapshot[applied={T|T < 7316102568695107584}]
I20260808 02:57:33.716907 6905 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:33.717206 6905 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:33.721179 6905 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 173e1a490a9e4f4e8962027bd525d3d3: Sending DeleteTablet for 3 replicas of tablet 5e577e29b09e49119e059178ef872428
I20260808 02:57:33.722445 7456 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e577e29b09e49119e059178ef872428 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:33 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:42752
I20260808 02:57:33.722633 7090 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e577e29b09e49119e059178ef872428 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:33 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:42224
I20260808 02:57:33.723294 7269 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e577e29b09e49119e059178ef872428 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:33 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:33630
I20260808 02:57:33.723704 7603 tablet_replica.cc:330] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: stopping tablet replica
I20260808 02:57:33.724223 7603 raft_consensus.cc:2229] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:33.724836 7603 raft_consensus.cc:2258] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:33.725617 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 6992
I20260808 02:57:33.727213 7603 ts_tablet_manager.cc:1905] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:33.734740 7603 ts_tablet_manager.cc:1918] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:33.735059 7603 log.cc:1196] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/5e577e29b09e49119e059178ef872428
I20260808 02:57:33.735747 7603 ts_tablet_manager.cc:1939] T 5e577e29b09e49119e059178ef872428 P 9012b175d7cf4087a3e8246fde62b153: Deleting consensus metadata
I20260808 02:57:33.738010 6888 catalog_manager.cc:4841] TS 9012b175d7cf4087a3e8246fde62b153 (127.0.205.131:44811): tablet 5e577e29b09e49119e059178ef872428 (table test-table [id=1a6b2b8f94394278af6712d265b59eaf]) successfully deleted
I20260808 02:57:33.743593 7605 tablet_replica.cc:330] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: stopping tablet replica
I20260808 02:57:33.744230 7605 raft_consensus.cc:2229] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:33.744928 7605 raft_consensus.cc:2258] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:33.746340 7604 tablet_replica.cc:330] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: stopping tablet replica
I20260808 02:57:33.746884 7604 raft_consensus.cc:2229] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:57:33.747712 7604 raft_consensus.cc:2258] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:33.747920 7605 ts_tablet_manager.cc:1905] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:33.749977 7604 ts_tablet_manager.cc:1905] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 02:57:33 dist-test-slave-6ntk krb5kdc[6848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:33.755522 7605 ts_tablet_manager.cc:1918] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:33.755728 7605 log.cc:1196] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/5e577e29b09e49119e059178ef872428
I20260808 02:57:33.756335 7605 ts_tablet_manager.cc:1939] T 5e577e29b09e49119e059178ef872428 P eb3901b865a64ede9d640e54857d6712: Deleting consensus metadata
I20260808 02:57:33.758289 7604 ts_tablet_manager.cc:1918] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:33.758550 7604 log.cc:1196] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/5e577e29b09e49119e059178ef872428
I20260808 02:57:33.758706 6885 catalog_manager.cc:4841] TS eb3901b865a64ede9d640e54857d6712 (127.0.205.130:35329): tablet 5e577e29b09e49119e059178ef872428 (table test-table [id=1a6b2b8f94394278af6712d265b59eaf]) successfully deleted
I20260808 02:57:33.759163 7604 ts_tablet_manager.cc:1939] T 5e577e29b09e49119e059178ef872428 P c0e8622914c44eba8bf269b464d7dd07: Deleting consensus metadata
I20260808 02:57:33.761174 6884 catalog_manager.cc:4841] TS c0e8622914c44eba8bf269b464d7dd07 (127.0.205.129:38459): tablet 5e577e29b09e49119e059178ef872428 (table test-table [id=1a6b2b8f94394278af6712d265b59eaf]) successfully deleted
I20260808 02:57:33.763803 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.729060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42256 (local address 127.0.205.129:38459)
0808 02:57:33.729666 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:33.729676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:33.742869 (+ 13193us) server_negotiation.cc:407] Connection header received
0808 02:57:33.743091 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:33.743100 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:33.743239 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:33.743403 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:33.746457 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.747722 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.749642 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.750420 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.750525 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.756279 (+ 5754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:33.756300 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:33.756327 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:33.756386 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:33.760492 (+ 4106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:33.761468 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:33.761477 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:33.761480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:33.761637 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:33.762133 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:33.762137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:33.762139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:33.762530 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:33.762851 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:33.763159 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 02:57:33.763416 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414}
I20260808 02:57:33.764554 7171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:33.867228 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 6992
I20260808 02:57:33.894349 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 7185
Aug 08 02:57:33 dist-test-slave-6ntk krb5kdc[6848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:57:33.910156 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:33.895166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33638 (local address 127.0.205.130:35329)
0808 02:57:33.895348 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:33.895352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:33.895372 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:57:33.895531 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:33.895535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:33.895600 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:33.895691 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:33.896919 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.897658 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.898937 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:33.899326 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:33.899467 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:33.903701 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:33.903718 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:33.903734 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:33.903782 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:33.907118 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:33.907952 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:33.907962 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:33.907964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:33.908116 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:33.908541 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:33.908544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:33.908546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:33.908964 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:33.909242 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:33.909636 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 02:57:33.909809 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:57:33.910871 7349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:34.002255 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 7185
I20260808 02:57:34.029109 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 7365
Aug 08 02:57:34 dist-test-slave-6ntk krb5kdc[6848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:57:34.045112 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:34.029999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.0.205.131:44811)
0808 02:57:34.030199 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:34.030207 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:34.030227 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:57:34.030467 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:34.030471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:34.030537 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:34.030650 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:34.032186 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:34.033038 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:34.034431 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:34.034821 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:34.034964 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:34.039519 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:34.039536 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:34.039554 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:34.039600 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:34.042696 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:34.043352 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:34.043360 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:34.043362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:34.043531 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:34.043860 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:34.043863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:34.043865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:34.044211 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:34.044460 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:34.044674 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 02:57:34.044842 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:57:34.045819 7539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:34.136945 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 7365
I20260808 02:57:34.165973 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 6867
I20260808 02:57:34.167373 6946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:34.286268 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 6867
2026-08-08T02:57:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4671 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T02:57:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:34Z Disabled control of system clock
I20260808 02:57:34.375710 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:44377
--webserver_interface=127.0.205.190
--webserver_port=0
--builtin_ntp_servers=127.0.205.148:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:44377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:57:34.720093 7618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:34.720412 7618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:34.720498 7618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:34.728016 7618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:34.728138 7618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:34.728178 7618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:34.728237 7618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:34.728281 7618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:34.737720 7618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:44377
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:44377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:34.739070 7618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:34.740909 7618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:34.753417 7626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:34.753449 7627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:34.753388 7629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:34.754432 7618 server_base.cc:1034] running on GCE node
I20260808 02:57:34.755309 7618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:34.756883 7618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:34.758131 7618 hybrid_clock.cc:648] HybridClock initialized: now 1786157854758086 us; error 60 us; skew 500 ppm
I20260808 02:57:34.758513 7618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:57:34.761101 7618 webserver.cc:460] Webserver started at http://127.0.205.190:46311/ using document root <none> and password file <none>
I20260808 02:57:34.761922 7618 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:34.762116 7618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:34.762490 7618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:34.765609 7618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "b67f4f625aae446eae02008f3c18f08f"
format_stamp: "Formatted at 2026-08-08 02:57:34 on dist-test-slave-6ntk"
server_key: "5f1698f35886d769d0cc1eab87fc6154"
server_key_iv: "5d1907073fd1bd77609f29c130524032"
server_key_version: "encryptionkey@0"
I20260808 02:57:34.766551 7618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "b67f4f625aae446eae02008f3c18f08f"
format_stamp: "Formatted at 2026-08-08 02:57:34 on dist-test-slave-6ntk"
server_key: "5f1698f35886d769d0cc1eab87fc6154"
server_key_iv: "5d1907073fd1bd77609f29c130524032"
server_key_version: "encryptionkey@0"
I20260808 02:57:34.771968 7618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:57:34.775746 7644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:34.777094 7618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:57:34.777289 7618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "b67f4f625aae446eae02008f3c18f08f"
format_stamp: "Formatted at 2026-08-08 02:57:34 on dist-test-slave-6ntk"
server_key: "5f1698f35886d769d0cc1eab87fc6154"
server_key_iv: "5d1907073fd1bd77609f29c130524032"
server_key_version: "encryptionkey@0"
I20260808 02:57:34.777524 7618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:34.783365 7618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:34.784171 7618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:34.784495 7618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:34.803241 7618 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:44377
I20260808 02:57:34.803261 7708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:44377 every 8 connection(s)
I20260808 02:57:34.804688 7618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:34.808035 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 7618
I20260808 02:57:34.808252 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:34.809029 7709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:34.809131 822 external_mini_cluster.cc:1428] Setting key 753cb2d972acfd43fae63481add64b7e
I20260808 02:57:34.817575 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:34.811307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58162 (local address 127.0.205.190:44377)
0808 02:57:34.812385 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:34.812400 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:34.812447 (+ 47us) server_negotiation.cc:407] Connection header received
0808 02:57:34.814037 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:34.814059 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:34.814505 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:34.814845 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:34.816012 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:34.816019 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:34.816031 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:34.816048 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:34.816107 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:34.816122 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:34.816302 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:34.816423 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 02:57:34.816698 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":402,"threads_started":1}
I20260808 02:57:34.822069 7709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f: Bootstrap starting.
I20260808 02:57:34.824934 7709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:34.826073 7709 log.cc:824] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:34.828665 7709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f: No bootstrap required, opened a new log
I20260808 02:57:34.835713 7709 raft_consensus.cc:348] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b67f4f625aae446eae02008f3c18f08f" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } }
I20260808 02:57:34.836081 7709 raft_consensus.cc:374] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:34.836192 7709 raft_consensus.cc:729] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b67f4f625aae446eae02008f3c18f08f, State: Initialized, Role: FOLLOWER
I20260808 02:57:34.836752 7709 consensus_queue.cc:260] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b67f4f625aae446eae02008f3c18f08f" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } }
I20260808 02:57:34.837005 7709 raft_consensus.cc:388] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:34.837157 7709 raft_consensus.cc:482] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:34.837335 7709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:34.839604 7709 raft_consensus.cc:504] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b67f4f625aae446eae02008f3c18f08f" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } }
I20260808 02:57:34.840116 7709 leader_election.cc:302] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b67f4f625aae446eae02008f3c18f08f; no voters:
I20260808 02:57:34.840715 7709 leader_election.cc:288] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:34.840853 7715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:34.841670 7715 raft_consensus.cc:686] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [term 1 LEADER]: Becoming Leader. State: Replica: b67f4f625aae446eae02008f3c18f08f, State: Running, Role: LEADER
I20260808 02:57:34.842291 7715 consensus_queue.cc:237] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b67f4f625aae446eae02008f3c18f08f" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } }
I20260808 02:57:34.842896 7709 sys_catalog.cc:564] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:34.845146 7718 sys_catalog.cc:455] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b67f4f625aae446eae02008f3c18f08f. Latest consensus state: current_term: 1 leader_uuid: "b67f4f625aae446eae02008f3c18f08f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b67f4f625aae446eae02008f3c18f08f" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } } }
I20260808 02:57:34.845549 7718 sys_catalog.cc:458] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:34.845153 7717 sys_catalog.cc:455] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b67f4f625aae446eae02008f3c18f08f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b67f4f625aae446eae02008f3c18f08f" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } } }
I20260808 02:57:34.847003 7717 sys_catalog.cc:458] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:34.851245 7721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:34.856889 7721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:34.866544 7721 catalog_manager.cc:1347] Generated new cluster ID: 1e2c07ca4ead4b828025dd78f891329a
I20260808 02:57:34.866689 7721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:34.882547 7721 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:34.882987 7721 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:34.893580 7721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f: Generated new TSK 0
I20260808 02:57:34.894819 7721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:57:34.903680 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--builtin_ntp_servers=127.0.205.148:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:57:35.257164 7741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:35.257481 7741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:35.257654 7741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:35.265183 7741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:35.265321 7741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:35.265543 7741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:35.275112 7741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:35.276628 7741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:35.278641 7741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:35.292613 7749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:35.292678 7752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:35.292589 7750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:35.293726 7741 server_base.cc:1034] running on GCE node
I20260808 02:57:35.294404 7741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:35.295456 7741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:35.296696 7741 hybrid_clock.cc:648] HybridClock initialized: now 1786157855296649 us; error 42 us; skew 500 ppm
I20260808 02:57:35.297020 7741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:57:35.299399 7741 webserver.cc:460] Webserver started at http://127.0.205.129:40325/ using document root <none> and password file <none>
I20260808 02:57:35.300112 7741 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:35.300241 7741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:35.300576 7741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:35.303579 7741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "1fc9433e6c8b4fba965a80c971baaa82"
format_stamp: "Formatted at 2026-08-08 02:57:35 on dist-test-slave-6ntk"
server_key: "eb4c056608439842ef11baa6e0b6be07"
server_key_iv: "069b8b6ecd63694d5d7d18f94540d4b2"
server_key_version: "encryptionkey@0"
I20260808 02:57:35.304412 7741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "1fc9433e6c8b4fba965a80c971baaa82"
format_stamp: "Formatted at 2026-08-08 02:57:35 on dist-test-slave-6ntk"
server_key: "eb4c056608439842ef11baa6e0b6be07"
server_key_iv: "069b8b6ecd63694d5d7d18f94540d4b2"
server_key_version: "encryptionkey@0"
I20260808 02:57:35.309741 7741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:57:35.313283 7760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:35.314651 7741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:35.314834 7741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "1fc9433e6c8b4fba965a80c971baaa82"
format_stamp: "Formatted at 2026-08-08 02:57:35 on dist-test-slave-6ntk"
server_key: "eb4c056608439842ef11baa6e0b6be07"
server_key_iv: "069b8b6ecd63694d5d7d18f94540d4b2"
server_key_version: "encryptionkey@0"
I20260808 02:57:35.315070 7741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:35.320971 7741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:35.321760 7741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:35.322094 7741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:35.322993 7741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:35.324424 7741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:35.324553 7741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:35.324682 7741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:35.324771 7741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:35.359751 7741 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:39047
I20260808 02:57:35.359805 7908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:39047 every 8 connection(s)
I20260808 02:57:35.361195 7741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:35.368256 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 7741
I20260808 02:57:35.368512 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:57:35.369786 822 external_mini_cluster.cc:1428] Setting key c1662f4c2269b268c53b908cca9c942d
I20260808 02:57:35.370229 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:35.364045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:57403 (local address 127.0.205.190:44377)
0808 02:57:35.364520 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:35.364527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:35.365807 (+ 1280us) server_negotiation.cc:407] Connection header received
0808 02:57:35.367590 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:35.367594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:35.367654 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:35.367754 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:35.369455 (+ 1701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:35.369459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:35.369462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:35.369478 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:35.369518 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:35.369530 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:35.369633 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:35.369927 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 02:57:35.370004 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":225,"threads_started":1}
I20260808 02:57:35.371522 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:35.364706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:44377 (local address 127.0.205.129:57403)
0808 02:57:35.365650 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:35.365688 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:35.367321 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:35.368087 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:35.368100 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:35.368242 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:35.369081 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:57:35.369086 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:35.369146 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:57:35.369267 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:35.369639 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:35.369652 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:35.369660 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:57:35.370005 (+ 345us) client_negotiation.cc:240] Negotiation successful
0808 02:57:35.370282 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":281,"threads_started":1}
I20260808 02:57:35.373050 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--builtin_ntp_servers=127.0.205.148:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 02:57:35.373070 7909 heartbeater.cc:347] Connected to a master server at 127.0.205.190:44377
I20260808 02:57:35.373407 7909 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:35.374148 7909 heartbeater.cc:511] Master 127.0.205.190:44377 requested a full tablet report, sending...
I20260808 02:57:35.376482 7667 ts_manager.cc:194] Registered new tserver with Master: 1fc9433e6c8b4fba965a80c971baaa82 (127.0.205.129:39047)
W20260808 02:57:35.714195 7916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:35.714493 7916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:35.714660 7916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:35.722088 7916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:35.722185 7916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:35.722402 7916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:35.731684 7916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:35.733062 7916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:35.734990 7916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:35.748807 7930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:35.748911 7925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:35.748747 7924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:35.750021 7916 server_base.cc:1034] running on GCE node
I20260808 02:57:35.750742 7916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:35.751653 7916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:35.752988 7916 hybrid_clock.cc:648] HybridClock initialized: now 1786157855752882 us; error 53 us; skew 500 ppm
I20260808 02:57:35.753473 7916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:57:35.755869 7916 webserver.cc:460] Webserver started at http://127.0.205.130:44133/ using document root <none> and password file <none>
I20260808 02:57:35.756654 7916 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:35.756781 7916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:35.757115 7916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:35.760119 7916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "dbfc19e799e1480dac7ac7f0c5612d97"
format_stamp: "Formatted at 2026-08-08 02:57:35 on dist-test-slave-6ntk"
server_key: "2a93f12f7aefd9cd568ca9b90697c0e9"
server_key_iv: "d3e1ab0ccde47c4fdeb0cd085dc26f7d"
server_key_version: "encryptionkey@0"
I20260808 02:57:35.760975 7916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "dbfc19e799e1480dac7ac7f0c5612d97"
format_stamp: "Formatted at 2026-08-08 02:57:35 on dist-test-slave-6ntk"
server_key: "2a93f12f7aefd9cd568ca9b90697c0e9"
server_key_iv: "d3e1ab0ccde47c4fdeb0cd085dc26f7d"
server_key_version: "encryptionkey@0"
I20260808 02:57:35.766505 7916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 02:57:35.770293 7937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:35.771711 7916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:57:35.771989 7916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "dbfc19e799e1480dac7ac7f0c5612d97"
format_stamp: "Formatted at 2026-08-08 02:57:35 on dist-test-slave-6ntk"
server_key: "2a93f12f7aefd9cd568ca9b90697c0e9"
server_key_iv: "d3e1ab0ccde47c4fdeb0cd085dc26f7d"
server_key_version: "encryptionkey@0"
I20260808 02:57:35.772238 7916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:35.778160 7916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:35.778990 7916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:35.779275 7916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:35.780201 7916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:35.781744 7916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:35.781869 7916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:35.782109 7916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:35.782218 7916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:35.817566 7916 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:45637
I20260808 02:57:35.817601 8094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:45637 every 8 connection(s)
I20260808 02:57:35.819175 7916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:35.827803 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:35.822023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:41169 (local address 127.0.205.190:44377)
0808 02:57:35.822247 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:35.822252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:35.823670 (+ 1418us) server_negotiation.cc:407] Connection header received
0808 02:57:35.825413 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:35.825419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:35.825511 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:35.825626 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:35.827031 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:35.827037 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:35.827042 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:35.827060 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:35.827102 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:35.827114 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:35.827239 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:35.827462 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 02:57:35.827535 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 02:57:35.828411 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 7916
I20260808 02:57:35.828578 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:57:35.828850 8098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:35.822603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:44377 (local address 127.0.205.130:41169)
0808 02:57:35.823510 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:35.823546 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:35.825140 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:35.825854 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:35.825861 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:35.826019 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:35.826733 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:57:35.826736 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:35.826773 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:57:35.826857 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:35.827222 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:35.827234 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:35.827241 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 02:57:35.827544 (+ 303us) client_negotiation.cc:240] Negotiation successful
0808 02:57:35.827787 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":314,"threads_started":1}
I20260808 02:57:35.829088 822 external_mini_cluster.cc:1428] Setting key 00b9db0550c5f3e77ca683932cbdeac3
I20260808 02:57:35.830369 8095 heartbeater.cc:347] Connected to a master server at 127.0.205.190:44377
I20260808 02:57:35.830675 8095 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:35.831401 8095 heartbeater.cc:511] Master 127.0.205.190:44377 requested a full tablet report, sending...
I20260808 02:57:35.832317 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--builtin_ntp_servers=127.0.205.148:35293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 02:57:35.832952 7667 ts_manager.cc:194] Registered new tserver with Master: dbfc19e799e1480dac7ac7f0c5612d97 (127.0.205.130:45637)
W20260808 02:57:36.172225 8099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:36.172528 8099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:36.172618 8099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:36.181516 8099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:36.181627 8099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:36.181841 8099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:36.194603 8099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:36.195977 8099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:36.197743 8099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:36.211481 8107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:36.211763 8106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:36.211519 8109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:36.212415 8099 server_base.cc:1034] running on GCE node
I20260808 02:57:36.213064 8099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:36.214119 8099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:36.215346 8099 hybrid_clock.cc:648] HybridClock initialized: now 1786157856215306 us; error 95 us; skew 500 ppm
I20260808 02:57:36.215667 8099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:57:36.218032 8099 webserver.cc:460] Webserver started at http://127.0.205.131:42553/ using document root <none> and password file <none>
I20260808 02:57:36.218858 8099 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:36.219027 8099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:36.219415 8099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:36.222443 8099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "411d368cd73746f5804b65c071626818"
format_stamp: "Formatted at 2026-08-08 02:57:36 on dist-test-slave-6ntk"
server_key: "17fcaafd8c395d4f18552689626e0f11"
server_key_iv: "876f45d446404bdf14d162ad84b6d716"
server_key_version: "encryptionkey@0"
I20260808 02:57:36.223279 8099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "411d368cd73746f5804b65c071626818"
format_stamp: "Formatted at 2026-08-08 02:57:36 on dist-test-slave-6ntk"
server_key: "17fcaafd8c395d4f18552689626e0f11"
server_key_iv: "876f45d446404bdf14d162ad84b6d716"
server_key_version: "encryptionkey@0"
I20260808 02:57:36.228678 8099 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:57:36.232331 8118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:36.233678 8099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:57:36.233915 8099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "411d368cd73746f5804b65c071626818"
format_stamp: "Formatted at 2026-08-08 02:57:36 on dist-test-slave-6ntk"
server_key: "17fcaafd8c395d4f18552689626e0f11"
server_key_iv: "876f45d446404bdf14d162ad84b6d716"
server_key_version: "encryptionkey@0"
I20260808 02:57:36.234185 8099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:36.240236 8099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:36.241092 8099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:36.241384 8099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:36.242576 8099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:36.244158 8099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:36.244284 8099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:36.244437 8099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:36.244536 8099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:36.281245 8099 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:34767
I20260808 02:57:36.281302 8283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:34767 every 8 connection(s)
I20260808 02:57:36.282761 8099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:57:36.286958 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 8099
I20260808 02:57:36.287145 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:57:36.287644 822 external_mini_cluster.cc:1428] Setting key 3dd680d7a6137765327f0ca34844253b
I20260808 02:57:36.291914 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.285496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:39759 (local address 127.0.205.190:44377)
0808 02:57:36.285678 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.285683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.287290 (+ 1607us) server_negotiation.cc:407] Connection header received
0808 02:57:36.289394 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.289400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.289475 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.289589 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.291263 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.291267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.291271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.291288 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.291322 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.291334 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.291437 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.291614 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.291668 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:57:36.292846 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.286128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:44377 (local address 127.0.205.131:39759)
0808 02:57:36.287097 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.287147 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:36.289087 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:36.289902 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:36.289917 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:36.290052 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:36.290923 (+ 871us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:57:36.290927 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:36.290988 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:57:36.291090 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:36.291430 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:36.291438 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:36.291443 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:57:36.291659 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 02:57:36.291851 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":297,"threads_started":1}
I20260808 02:57:36.294384 8284 heartbeater.cc:347] Connected to a master server at 127.0.205.190:44377
I20260808 02:57:36.294700 8284 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:36.295301 8284 heartbeater.cc:511] Master 127.0.205.190:44377 requested a full tablet report, sending...
I20260808 02:57:36.296852 7667 ts_manager.cc:194] Registered new tserver with Master: 411d368cd73746f5804b65c071626818 (127.0.205.131:34767)
I20260808 02:57:36.303354 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:57:36.310636 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.308137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.0.205.190:44377)
0808 02:57:36.308333 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.308338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.308863 (+ 525us) server_negotiation.cc:407] Connection header received
0808 02:57:36.309015 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.309018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.309073 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.309151 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.310104 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.310108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.310111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.310126 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.310159 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.310170 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.310271 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.310366 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.310424 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:57:36.314834 7667 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:57:36.321743 7667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:36.341790 8302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.334312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:45637 (local address 127.0.0.1:35248)
0808 02:57:36.335171 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.335209 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:36.335344 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:36.338172 (+ 2828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:36.338184 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:36.338319 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:36.339327 (+ 1008us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:57:36.339334 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:36.339380 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:57:36.339466 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:36.340951 (+ 1485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:36.340964 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:36.340972 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:57:36.341166 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 02:57:36.341268 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":470,"threads_started":1}
I20260808 02:57:36.343014 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.334741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35248 (local address 127.0.205.130:45637)
0808 02:57:36.337408 (+ 2667us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.337420 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.337459 (+ 39us) server_negotiation.cc:407] Connection header received
0808 02:57:36.337555 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.337567 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.337861 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.339094 (+ 1233us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.340685 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.340693 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.340707 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.340731 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.340790 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.340816 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.342394 (+ 1578us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.342481 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.342593 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2461,"thread_start_us":2295,"threads_started":1}
I20260808 02:57:36.343668 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.336687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37760 (local address 127.0.205.129:39047)
0808 02:57:36.340013 (+ 3326us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.340021 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.340426 (+ 405us) server_negotiation.cc:407] Connection header received
0808 02:57:36.340513 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.340523 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.340733 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.340910 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.342230 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.342238 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.342252 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.342270 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.342617 (+ 347us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.342663 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.342799 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.343244 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.343353 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3141,"thread_start_us":3008,"threads_started":1}
I20260808 02:57:36.344290 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.334985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41306 (local address 127.0.205.131:34767)
0808 02:57:36.336860 (+ 1875us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.336873 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.340061 (+ 3188us) server_negotiation.cc:407] Connection header received
0808 02:57:36.340530 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.340539 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.340719 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.340815 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.343116 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.343124 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.343136 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.343156 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.343211 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.343232 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.343382 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.343657 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.343753 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1651,"thread_start_us":2147,"threads_started":1}
I20260808 02:57:36.344946 8303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.334507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:34767 (local address 127.0.0.1:41306)
0808 02:57:36.339921 (+ 5414us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.339946 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:36.340345 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:36.342001 (+ 1656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:36.342005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:36.342026 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:36.342819 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:57:36.342824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:36.342842 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:57:36.342882 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:36.343495 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:36.343501 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:36.343505 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:57:36.344609 (+ 1104us) client_negotiation.cc:240] Negotiation successful
0808 02:57:36.344666 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5276,"thread_start_us":4387,"threads_started":1}
I20260808 02:57:36.345263 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.337071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:39047 (local address 127.0.0.1:37760)
0808 02:57:36.340292 (+ 3221us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.340309 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:36.340404 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:36.340931 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:36.340935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:36.340959 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:36.341743 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:57:36.341746 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:36.341765 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:57:36.341824 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:36.342858 (+ 1034us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:36.342863 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:36.342867 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:57:36.343046 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 02:57:36.343850 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3104,"thread_start_us":3022,"threads_started":1}
I20260808 02:57:36.347277 8001 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3a2902e31d4760a2cc2634cf658f46 (DEFAULT_TABLE table=test-table [id=42a24faf717f4387a3ceeb811dd16eb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:36.348738 8001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3a2902e31d4760a2cc2634cf658f46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:36.348723 8191 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3a2902e31d4760a2cc2634cf658f46 (DEFAULT_TABLE table=test-table [id=42a24faf717f4387a3ceeb811dd16eb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:36.348723 7824 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3a2902e31d4760a2cc2634cf658f46 (DEFAULT_TABLE table=test-table [id=42a24faf717f4387a3ceeb811dd16eb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:36.350194 8191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3a2902e31d4760a2cc2634cf658f46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:36.350183 7824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3a2902e31d4760a2cc2634cf658f46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:36.367704 8310 tablet_bootstrap.cc:492] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: Bootstrap starting.
I20260808 02:57:36.371080 8314 tablet_bootstrap.cc:492] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: Bootstrap starting.
I20260808 02:57:36.373689 8310 tablet_bootstrap.cc:654] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:36.375430 8310 log.cc:824] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:36.375721 8312 tablet_bootstrap.cc:492] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82: Bootstrap starting.
I20260808 02:57:36.376209 8314 tablet_bootstrap.cc:654] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:36.377599 8314 log.cc:824] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:36.379415 8312 tablet_bootstrap.cc:654] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:36.379776 8310 tablet_bootstrap.cc:492] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: No bootstrap required, opened a new log
I20260808 02:57:36.379720 7909 heartbeater.cc:503] Master 127.0.205.190:44377 was elected leader, sending a full tablet report...
I20260808 02:57:36.380111 8310 ts_tablet_manager.cc:1397] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260808 02:57:36.380918 8314 tablet_bootstrap.cc:492] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: No bootstrap required, opened a new log
I20260808 02:57:36.381183 8314 ts_tablet_manager.cc:1397] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 02:57:36.381148 8312 log.cc:824] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:36.387339 8312 tablet_bootstrap.cc:492] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82: No bootstrap required, opened a new log
I20260808 02:57:36.387629 8312 ts_tablet_manager.cc:1397] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.001s
I20260808 02:57:36.391863 8310 raft_consensus.cc:348] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 45637 } } peers { permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 } } peers { permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 } }
I20260808 02:57:36.391863 8314 raft_consensus.cc:348] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 45637 } } peers { permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 } } peers { permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 } }
I20260808 02:57:36.392330 8310 raft_consensus.cc:374] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:36.392329 8314 raft_consensus.cc:374] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:36.392438 8314 raft_consensus.cc:729] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 411d368cd73746f5804b65c071626818, State: Initialized, Role: FOLLOWER
I20260808 02:57:36.392438 8310 raft_consensus.cc:729] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbfc19e799e1480dac7ac7f0c5612d97, State: Initialized, Role: FOLLOWER
I20260808 02:57:36.393090 8314 consensus_queue.cc:260] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 45637 } } peers { permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 } } peers { permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 } }
I20260808 02:57:36.393088 8310 consensus_queue.cc:260] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 45637 } } peers { permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 } } peers { permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 } }
I20260808 02:57:36.394762 8095 heartbeater.cc:503] Master 127.0.205.190:44377 was elected leader, sending a full tablet report...
I20260808 02:57:36.394824 8284 heartbeater.cc:503] Master 127.0.205.190:44377 was elected leader, sending a full tablet report...
I20260808 02:57:36.395140 8314 ts_tablet_manager.cc:1428] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260808 02:57:36.396031 8310 ts_tablet_manager.cc:1428] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260808 02:57:36.395562 8312 raft_consensus.cc:348] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 45637 } } peers { permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 } } peers { permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 } }
I20260808 02:57:36.397018 8312 raft_consensus.cc:374] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:36.397137 8312 raft_consensus.cc:729] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc9433e6c8b4fba965a80c971baaa82, State: Initialized, Role: FOLLOWER
I20260808 02:57:36.397841 8312 consensus_queue.cc:260] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 45637 } } peers { permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 } } peers { permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 } }
I20260808 02:57:36.399781 8312 ts_tablet_manager.cc:1428] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260808 02:57:36.431540 8320 raft_consensus.cc:482] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:36.431915 8320 raft_consensus.cc:504] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 45637 } } peers { permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 } } peers { permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 } }
I20260808 02:57:36.433696 8320 leader_election.cc:288] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 411d368cd73746f5804b65c071626818 (127.0.205.131:34767), 1fc9433e6c8b4fba965a80c971baaa82 (127.0.205.129:39047)
I20260808 02:57:36.437949 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.434275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:41771 (local address 127.0.205.131:34767)
0808 02:57:36.434490 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.434494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.435694 (+ 1200us) server_negotiation.cc:407] Connection header received
0808 02:57:36.435864 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.435877 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.435927 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.436033 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.437246 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.437252 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.437257 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.437282 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.437313 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.437325 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.437463 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.437548 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.437618 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:57:36.438002 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.434133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:41269 (local address 127.0.205.129:39047)
0808 02:57:36.434318 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.434344 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.435635 (+ 1291us) server_negotiation.cc:407] Connection header received
0808 02:57:36.435815 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.435824 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.435888 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.436006 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.437193 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.437268 (+ 75us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.437271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.437291 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.437325 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.437339 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.437504 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.437568 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.437651 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:57:36.438298 8326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.434292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:39047 (local address 127.0.205.130:41269)
0808 02:57:36.435514 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.435532 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:36.435654 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:36.436038 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:36.436042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:36.436068 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:36.436967 (+ 899us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:57:36.436969 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:36.436989 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:57:36.437032 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:36.437452 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:36.437458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:36.437462 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:57:36.437732 (+ 270us) client_negotiation.cc:240] Negotiation successful
0808 02:57:36.437791 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"thread_start_us":520,"threads_started":1}
I20260808 02:57:36.438573 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.434576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:34767 (local address 127.0.205.130:41771)
0808 02:57:36.435518 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.435537 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:36.435732 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:36.436057 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:36.436061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:36.436085 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:36.437048 (+ 963us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:57:36.437051 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:36.437063 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:57:36.437099 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:36.437437 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:36.437444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:36.437449 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:57:36.437612 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 02:57:36.437663 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"spinlock_wait_cycles":25344,"thread_start_us":495,"threads_started":1}
I20260808 02:57:36.439396 7851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3a2902e31d4760a2cc2634cf658f46" candidate_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc9433e6c8b4fba965a80c971baaa82" is_pre_election: true
I20260808 02:57:36.439677 8221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3a2902e31d4760a2cc2634cf658f46" candidate_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "411d368cd73746f5804b65c071626818" is_pre_election: true
I20260808 02:57:36.440093 7851 raft_consensus.cc:2454] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbfc19e799e1480dac7ac7f0c5612d97 in term 0.
I20260808 02:57:36.440169 8221 raft_consensus.cc:2454] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbfc19e799e1480dac7ac7f0c5612d97 in term 0.
I20260808 02:57:36.440922 7940 leader_election.cc:302] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1fc9433e6c8b4fba965a80c971baaa82, 411d368cd73746f5804b65c071626818, dbfc19e799e1480dac7ac7f0c5612d97; no voters:
I20260808 02:57:36.441367 8320 raft_consensus.cc:2781] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:36.441542 8320 raft_consensus.cc:482] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:36.441679 8320 raft_consensus.cc:3037] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:36.444221 8320 raft_consensus.cc:504] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 45637 } } peers { permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 } } peers { permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 } }
I20260808 02:57:36.444980 8320 leader_election.cc:288] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [CANDIDATE]: Term 1 election: Requested vote from peers 411d368cd73746f5804b65c071626818 (127.0.205.131:34767), 1fc9433e6c8b4fba965a80c971baaa82 (127.0.205.129:39047)
I20260808 02:57:36.445354 8221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3a2902e31d4760a2cc2634cf658f46" candidate_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "411d368cd73746f5804b65c071626818"
I20260808 02:57:36.445552 7851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3a2902e31d4760a2cc2634cf658f46" candidate_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc9433e6c8b4fba965a80c971baaa82"
I20260808 02:57:36.445598 8221 raft_consensus.cc:3037] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:36.445770 7851 raft_consensus.cc:3037] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:36.448104 7851 raft_consensus.cc:2454] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbfc19e799e1480dac7ac7f0c5612d97 in term 1.
I20260808 02:57:36.448115 8221 raft_consensus.cc:2454] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbfc19e799e1480dac7ac7f0c5612d97 in term 1.
I20260808 02:57:36.448544 7940 leader_election.cc:302] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc9433e6c8b4fba965a80c971baaa82, dbfc19e799e1480dac7ac7f0c5612d97; no voters:
I20260808 02:57:36.448925 8320 raft_consensus.cc:2781] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:36.449460 8320 raft_consensus.cc:686] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 1 LEADER]: Becoming Leader. State: Replica: dbfc19e799e1480dac7ac7f0c5612d97, State: Running, Role: LEADER
I20260808 02:57:36.450071 8320 consensus_queue.cc:237] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 45637 } } peers { permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 } } peers { permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 } }
I20260808 02:57:36.455052 7665 catalog_manager.cc:5495] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbfc19e799e1480dac7ac7f0c5612d97 (127.0.205.130). New cstate: current_term: 1 leader_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbfc19e799e1480dac7ac7f0c5612d97" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 45637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 } health_report { overall_health: UNKNOWN } } }
I20260808 02:57:36.479349 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.477052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35260 (local address 127.0.205.130:45637)
0808 02:57:36.477228 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.477233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.477387 (+ 154us) server_negotiation.cc:407] Connection header received
0808 02:57:36.477534 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.477538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.477625 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.477720 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.478716 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.478722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.478725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.478741 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.478776 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.478788 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.478886 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.478971 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.479017 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:57:36.488341 8221 raft_consensus.cc:1261] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818 [term 1 FOLLOWER]: Refusing update from remote peer dbfc19e799e1480dac7ac7f0c5612d97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:36.488363 7851 raft_consensus.cc:1261] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82 [term 1 FOLLOWER]: Refusing update from remote peer dbfc19e799e1480dac7ac7f0c5612d97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:36.489409 8328 consensus_queue.cc:1035] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "411d368cd73746f5804b65c071626818" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 34767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:36.489782 8320 consensus_queue.cc:1035] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc9433e6c8b4fba965a80c971baaa82" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 39047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:36.520274 7665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:36.520620 7665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:36.524310 7665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b67f4f625aae446eae02008f3c18f08f: Sending DeleteTablet for 3 replicas of tablet 5b3a2902e31d4760a2cc2634cf658f46
I20260808 02:57:36.525673 8001 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3a2902e31d4760a2cc2634cf658f46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:36 UTC) from {username='slave'} at 127.0.0.1:35248
I20260808 02:57:36.525763 8191 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3a2902e31d4760a2cc2634cf658f46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:36 UTC) from {username='slave'} at 127.0.0.1:41306
I20260808 02:57:36.525751 7824 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3a2902e31d4760a2cc2634cf658f46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:36 UTC) from {username='slave'} at 127.0.0.1:37760
I20260808 02:57:36.527622 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 7741
I20260808 02:57:36.541105 8347 tablet_replica.cc:330] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82: stopping tablet replica
I20260808 02:57:36.541677 8347 raft_consensus.cc:2229] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:36.542317 8346 tablet_replica.cc:330] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: stopping tablet replica
I20260808 02:57:36.542316 8347 raft_consensus.cc:2258] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:36.542759 8346 raft_consensus.cc:2229] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:57:36.543499 8346 raft_consensus.cc:2258] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:36.545027 8347 ts_tablet_manager.cc:1905] T 5b3a2902e31d4760a2cc2634cf658f46 P 1fc9433e6c8b4fba965a80c971baaa82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:36.545275 8345 tablet_replica.cc:330] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: stopping tablet replica
I20260808 02:57:36.545653 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.529397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37774 (local address 127.0.205.129:39047)
0808 02:57:36.529635 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.529644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.540280 (+ 10636us) server_negotiation.cc:407] Connection header received
0808 02:57:36.540913 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.540922 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.541044 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.541142 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.544696 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.544702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.544707 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.544730 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.544772 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.544788 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.544934 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.545266 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.545340 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:57:36.545709 8345 raft_consensus.cc:2229] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:36.546254 8345 raft_consensus.cc:2258] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:36.546713 7903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:36.547703 8346 ts_tablet_manager.cc:1905] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:36.548588 8345 ts_tablet_manager.cc:1905] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:36.555955 8346 ts_tablet_manager.cc:1918] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:36.555943 8345 ts_tablet_manager.cc:1918] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:36.556147 8346 log.cc:1196] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/5b3a2902e31d4760a2cc2634cf658f46
I20260808 02:57:36.556146 8345 log.cc:1196] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/5b3a2902e31d4760a2cc2634cf658f46
I20260808 02:57:36.556733 8346 ts_tablet_manager.cc:1939] T 5b3a2902e31d4760a2cc2634cf658f46 P dbfc19e799e1480dac7ac7f0c5612d97: Deleting consensus metadata
I20260808 02:57:36.556782 8345 ts_tablet_manager.cc:1939] T 5b3a2902e31d4760a2cc2634cf658f46 P 411d368cd73746f5804b65c071626818: Deleting consensus metadata
I20260808 02:57:36.558967 7646 catalog_manager.cc:4841] TS 411d368cd73746f5804b65c071626818 (127.0.205.131:34767): tablet 5b3a2902e31d4760a2cc2634cf658f46 (table test-table [id=42a24faf717f4387a3ceeb811dd16eb6]) successfully deleted
I20260808 02:57:36.559198 7649 catalog_manager.cc:4841] TS dbfc19e799e1480dac7ac7f0c5612d97 (127.0.205.130:45637): tablet 5b3a2902e31d4760a2cc2634cf658f46 (table test-table [id=42a24faf717f4387a3ceeb811dd16eb6]) successfully deleted
I20260808 02:57:36.650152 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 7741
I20260808 02:57:36.678792 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 7916
W20260808 02:57:36.678797 7648 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.205.129:39047 (error 108)
I20260808 02:57:36.681368 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.680804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:39047 (local address 127.0.0.1:37778)
0808 02:57:36.681019 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.681177 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 02:57:36.681630 7648 catalog_manager.cc:4584] TS 1fc9433e6c8b4fba965a80c971baaa82 (127.0.205.129:39047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b3a2902e31d4760a2cc2634cf658f46: Network error: Client connection negotiation failed: client connection to 127.0.205.129:39047: connect: Connection refused (error 111)
I20260808 02:57:36.682804 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.679878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.0.205.130:45637)
0808 02:57:36.680070 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.680077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.680103 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:57:36.680329 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.680336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.680405 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.680535 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.681676 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.681680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.681683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.681699 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.681731 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.681745 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.681945 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.682400 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.682470 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:57:36.683847 8088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:36.735540 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.734965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:39047 (local address 127.0.0.1:37782)
0808 02:57:36.735216 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.735328 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 02:57:36.736689 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.736314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:39047 (local address 127.0.0.1:37794)
0808 02:57:36.736496 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.736571 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 02:57:36.797570 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 7916
I20260808 02:57:36.798020 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.797311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:39047 (local address 127.0.0.1:37808)
0808 02:57:36.797630 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.797772 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260808 02:57:36.800222 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.799752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:39047 (local address 127.0.0.1:37822)
0808 02:57:36.799930 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.800015 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 02:57:36.832233 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 8099
I20260808 02:57:36.835575 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.833190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.0.205.131:34767)
0808 02:57:36.833395 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:36.833402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:36.833428 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:57:36.833580 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:36.833587 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:36.833692 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:36.833819 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:36.834855 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:36.834862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:36.834868 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:36.834888 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:36.834941 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:36.834958 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:36.835107 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:36.835170 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 02:57:36.835233 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:57:36.836354 8279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:36.881742 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.881126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:39047 (local address 127.0.0.1:37824)
0808 02:57:36.881425 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.881553 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260808 02:57:36.883117 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:36.882643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:39047 (local address 127.0.0.1:37826)
0808 02:57:36.882881 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:36.882970 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260808 02:57:36.928290 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 8099
I20260808 02:57:36.954416 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 7618
I20260808 02:57:36.955701 7700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:37.066179 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:37.065481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:39047 (local address 127.0.0.1:37830)
0808 02:57:37.065675 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:37.065794 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:39047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 02:57:37.066391 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 7618
2026-08-08T02:57:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2773 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 02:57:37.115360 822 mini_oidc.cc:152] Starting JWKS server
I20260808 02:57:37.118409 822 webserver.cc:460] Webserver started at https://127.0.0.1:44115/ using document root <none> and password file <none>
I20260808 02:57:37.118793 822 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:57:37.119488 822 webserver.cc:460] Webserver started at http://127.0.0.1:41511/ using document root <none> and password file <none>
2026-08-08T02:57:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:37Z Disabled control of system clock
I20260808 02:57:37.155507 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:39491
--webserver_interface=127.0.205.190
--webserver_port=0
--builtin_ntp_servers=127.0.205.148:37585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:39491
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44115/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/testchainca.cert with env {}
W20260808 02:57:37.505124 8365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:37.505518 8365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:37.505686 8365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:37.513259 8365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:37.513432 8365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:37.513531 8365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:37.513643 8365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:37.513736 8365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:57:37.513830 8365 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:57:37.513939 8365 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44115/jwks.json
I20260808 02:57:37.524737 8365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:37585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:39491
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44115/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:37.526265 8365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:37.527647 8365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:37.540091 8375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:37.540174 8372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:37.540170 8373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:37.541101 8365 server_base.cc:1034] running on GCE node
I20260808 02:57:37.542089 8365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:37.543609 8365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:37.544880 8365 hybrid_clock.cc:648] HybridClock initialized: now 1786157857544816 us; error 56 us; skew 500 ppm
I20260808 02:57:37.545289 8365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:57:37.547899 8365 webserver.cc:460] Webserver started at http://127.0.205.190:38435/ using document root <none> and password file <none>
I20260808 02:57:37.548666 8365 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:37.548830 8365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:37.549182 8365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:37.552295 8365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "cf69bed5d90048e5a972afc0ee7db120"
format_stamp: "Formatted at 2026-08-08 02:57:37 on dist-test-slave-6ntk"
server_key: "159d28f87996fbdef195b9a43ca7890f"
server_key_iv: "215c96cb4215989dbe850a48b71061fe"
server_key_version: "encryptionkey@0"
I20260808 02:57:37.553133 8365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "cf69bed5d90048e5a972afc0ee7db120"
format_stamp: "Formatted at 2026-08-08 02:57:37 on dist-test-slave-6ntk"
server_key: "159d28f87996fbdef195b9a43ca7890f"
server_key_iv: "215c96cb4215989dbe850a48b71061fe"
server_key_version: "encryptionkey@0"
I20260808 02:57:37.558727 8365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 02:57:37.562495 8382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:37.563912 8365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 02:57:37.564118 8365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "cf69bed5d90048e5a972afc0ee7db120"
format_stamp: "Formatted at 2026-08-08 02:57:37 on dist-test-slave-6ntk"
server_key: "159d28f87996fbdef195b9a43ca7890f"
server_key_iv: "215c96cb4215989dbe850a48b71061fe"
server_key_version: "encryptionkey@0"
I20260808 02:57:37.564375 8365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:37.600421 8365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:37.601374 8365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:37.601740 8365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:37.621881 8365 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:39491
I20260808 02:57:37.621953 8451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:39491 every 8 connection(s)
I20260808 02:57:37.623471 8365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:37.627492 8452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:37.629990 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 8365
I20260808 02:57:37.630231 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:37.630894 822 external_mini_cluster.cc:1428] Setting key 3fb702d253bcd1f4dbbf938e168da325
I20260808 02:57:37.641515 8452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120: Bootstrap starting.
I20260808 02:57:37.645758 8452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:37.647275 8452 log.cc:824] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:37.648034 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:37.633038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.0.205.190:39491)
0808 02:57:37.634231 (+ 1193us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:37.634248 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:37.634299 (+ 51us) server_negotiation.cc:407] Connection header received
0808 02:57:37.636764 (+ 2465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:37.636796 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:37.637505 (+ 709us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:37.638042 (+ 537us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:37.638633 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:37.640649 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:37.642245 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:37.643311 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:37.643554 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:37.644827 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:37.644837 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:37.644855 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:37.645578 (+ 723us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:37.645721 (+ 143us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:37.645764 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:37.645939 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:37.646247 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 02:57:37.646609 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":331,"threads_started":1}
I20260808 02:57:37.650800 8452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120: No bootstrap required, opened a new log
I20260808 02:57:37.652441 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:57:37.659266 8452 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf69bed5d90048e5a972afc0ee7db120" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39491 } }
I20260808 02:57:37.659690 8452 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:37.659801 8452 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf69bed5d90048e5a972afc0ee7db120, State: Initialized, Role: FOLLOWER
I20260808 02:57:37.660485 8452 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf69bed5d90048e5a972afc0ee7db120" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39491 } }
I20260808 02:57:37.660805 8452 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:37.660964 8452 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:37.661171 8452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:37.663525 8452 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf69bed5d90048e5a972afc0ee7db120" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39491 } }
I20260808 02:57:37.664074 8452 leader_election.cc:302] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf69bed5d90048e5a972afc0ee7db120; no voters:
I20260808 02:57:37.664727 8452 leader_election.cc:288] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:37.664872 8458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:37.665738 8458 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [term 1 LEADER]: Becoming Leader. State: Replica: cf69bed5d90048e5a972afc0ee7db120, State: Running, Role: LEADER
I20260808 02:57:37.666307 8458 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf69bed5d90048e5a972afc0ee7db120" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39491 } }
I20260808 02:57:37.667165 8452 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:37.669243 8459 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf69bed5d90048e5a972afc0ee7db120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf69bed5d90048e5a972afc0ee7db120" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39491 } } }
I20260808 02:57:37.671303 8459 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:37.670779 8461 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf69bed5d90048e5a972afc0ee7db120. Latest consensus state: current_term: 1 leader_uuid: "cf69bed5d90048e5a972afc0ee7db120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf69bed5d90048e5a972afc0ee7db120" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39491 } } }
I20260808 02:57:37.671651 8461 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:37.675812 8468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:37.681639 8468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:37.692689 8468 catalog_manager.cc:1347] Generated new cluster ID: 5c8a05efc70a461397f424361f742b5d
I20260808 02:57:37.692821 8468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:37.724311 8468 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:37.725768 8468 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:37.733044 8468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf69bed5d90048e5a972afc0ee7db120: Generated new TSK 0
I20260808 02:57:37.734210 8468 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 02:57:37.785454 8497 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 02:57:37.785729 8497 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:57:37.786885 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:37.763729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.0.205.190:39491)
0808 02:57:37.764012 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:37.764022 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:37.764535 (+ 513us) server_negotiation.cc:407] Connection header received
0808 02:57:37.765025 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:37.765029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:37.765110 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:37.765240 (+ 130us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:57:37.765999 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:37.766764 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:37.769072 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:37.769681 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:37.769829 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:37.770620 (+ 791us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:57:37.786291 (+ 15671us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 02:57:37.786623 (+ 332us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41406: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":94}
W20260808 02:57:37.787042 8456 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41406: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 02:57:37.789229 8496 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:57:37.763642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:39491 (local address 127.0.0.1:41406)
0808 02:57:37.764406 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:37.764430 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:37.764654 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:37.765320 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:37.765326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:37.765340 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:57:37.765797 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:37.765810 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:37.766923 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:37.766930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:37.768813 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:37.768836 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:37.769906 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:37.769915 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:37.770119 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:37.770216 (+ 97us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:57:37.786643 (+ 16427us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 02:57:37.787056 (+ 413us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.205.190:39491: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":372,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.205.190:39491: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 02:57:37.791817 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 8365
I20260808 02:57:37.793068 8443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:37.891081 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 8365
2026-08-08T02:57:37Z chronyd exiting
I20260808 02:57:37.949649 822 test_util.cc:175] -----------------------------------------------
I20260808 02:57:37.949795 822 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786157811617362-822-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (839 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 02:57:37.953622 822 mini_oidc.cc:152] Starting JWKS server
I20260808 02:57:37.955595 822 webserver.cc:460] Webserver started at https://127.0.0.1:38777/ using document root <none> and password file <none>
I20260808 02:57:37.955780 822 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:57:37.956379 822 webserver.cc:460] Webserver started at http://127.0.0.1:39771/ using document root <none> and password file <none>
2026-08-08T02:57:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:37Z Disabled control of system clock
I20260808 02:57:37.990168 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:42641
--webserver_interface=127.0.205.190
--webserver_port=0
--builtin_ntp_servers=127.0.205.148:39995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:42641
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38777/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/testchainca.cert with env {}
W20260808 02:57:38.338024 8507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:38.338423 8507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:38.338605 8507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:38.346020 8507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:38.346167 8507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:38.346243 8507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:38.346334 8507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:38.346423 8507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:57:38.346517 8507 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:57:38.346598 8507 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38777/jwks.json
I20260808 02:57:38.357188 8507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:42641
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:42641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38777/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:38.358737 8507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:38.360137 8507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:38.372825 8513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:38.372768 8516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:38.372792 8514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:38.373725 8507 server_base.cc:1034] running on GCE node
I20260808 02:57:38.374621 8507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:38.376150 8507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:38.377411 8507 hybrid_clock.cc:648] HybridClock initialized: now 1786157858377363 us; error 58 us; skew 500 ppm
I20260808 02:57:38.377792 8507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:57:38.380295 8507 webserver.cc:460] Webserver started at http://127.0.205.190:36231/ using document root <none> and password file <none>
I20260808 02:57:38.381031 8507 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:38.381182 8507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:38.381527 8507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:38.384506 8507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "7928e02b08b1448c9a1bc59a91d872b4"
format_stamp: "Formatted at 2026-08-08 02:57:38 on dist-test-slave-6ntk"
server_key: "936395a7d10c93776eaaa415a5950b61"
server_key_iv: "2694036ef0933081545fc29f4c315e00"
server_key_version: "encryptionkey@0"
I20260808 02:57:38.385336 8507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "7928e02b08b1448c9a1bc59a91d872b4"
format_stamp: "Formatted at 2026-08-08 02:57:38 on dist-test-slave-6ntk"
server_key: "936395a7d10c93776eaaa415a5950b61"
server_key_iv: "2694036ef0933081545fc29f4c315e00"
server_key_version: "encryptionkey@0"
I20260808 02:57:38.391847 8507 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260808 02:57:38.395776 8528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:38.397331 8507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:57:38.397562 8507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "7928e02b08b1448c9a1bc59a91d872b4"
format_stamp: "Formatted at 2026-08-08 02:57:38 on dist-test-slave-6ntk"
server_key: "936395a7d10c93776eaaa415a5950b61"
server_key_iv: "2694036ef0933081545fc29f4c315e00"
server_key_version: "encryptionkey@0"
I20260808 02:57:38.397802 8507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:38.433763 8507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:38.434868 8507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:38.435220 8507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:38.454669 8507 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:42641
I20260808 02:57:38.454689 8598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:42641 every 8 connection(s)
I20260808 02:57:38.456139 8507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:38.460160 8599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:38.462765 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 8507
I20260808 02:57:38.462975 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:38.463809 822 external_mini_cluster.cc:1428] Setting key b949bf8dfb26b95d44808e3f8fbf214b
I20260808 02:57:38.473568 8599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4: Bootstrap starting.
I20260808 02:57:38.477463 8599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:38.478690 8599 log.cc:824] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:38.479203 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:38.466159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.0.205.190:42641)
0808 02:57:38.467429 (+ 1270us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:38.467441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:38.467490 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:57:38.469495 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:38.469524 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:38.470209 (+ 685us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:38.470626 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:38.471258 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:38.472581 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:38.474406 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:38.475100 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:38.475246 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:38.476589 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:38.476597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:38.476611 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:38.477186 (+ 575us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:38.477291 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:38.477313 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:38.477420 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:38.477730 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 02:57:38.478136 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":493,"threads_started":1}
I20260808 02:57:38.481688 8599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4: No bootstrap required, opened a new log
I20260808 02:57:38.483266 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:57:38.490234 8599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7928e02b08b1448c9a1bc59a91d872b4" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42641 } }
I20260808 02:57:38.490571 8599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:38.490658 8599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7928e02b08b1448c9a1bc59a91d872b4, State: Initialized, Role: FOLLOWER
I20260808 02:57:38.491297 8599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7928e02b08b1448c9a1bc59a91d872b4" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42641 } }
I20260808 02:57:38.491544 8599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:38.491704 8599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:38.491869 8599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:38.494418 8599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7928e02b08b1448c9a1bc59a91d872b4" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42641 } }
I20260808 02:57:38.494951 8599 leader_election.cc:302] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7928e02b08b1448c9a1bc59a91d872b4; no voters:
I20260808 02:57:38.495548 8599 leader_election.cc:288] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:38.495716 8604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:38.496763 8604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [term 1 LEADER]: Becoming Leader. State: Replica: 7928e02b08b1448c9a1bc59a91d872b4, State: Running, Role: LEADER
I20260808 02:57:38.497673 8599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:38.497650 8604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7928e02b08b1448c9a1bc59a91d872b4" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42641 } }
I20260808 02:57:38.500914 8606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7928e02b08b1448c9a1bc59a91d872b4. Latest consensus state: current_term: 1 leader_uuid: "7928e02b08b1448c9a1bc59a91d872b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7928e02b08b1448c9a1bc59a91d872b4" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42641 } } }
I20260808 02:57:38.500871 8605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7928e02b08b1448c9a1bc59a91d872b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7928e02b08b1448c9a1bc59a91d872b4" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42641 } } }
I20260808 02:57:38.501471 8606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:38.501469 8605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:38.504693 8618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:38.510332 8618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:38.520366 8618 catalog_manager.cc:1347] Generated new cluster ID: 8d31aa3b64254a02958318babe8d0b0b
I20260808 02:57:38.520529 8618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:38.535857 8618 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:38.537212 8618 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:38.545760 8618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7928e02b08b1448c9a1bc59a91d872b4: Generated new TSK 0
I20260808 02:57:38.546778 8618 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 02:57:38.581700 8636 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 02:57:38.581892 8636 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:57:38.583371 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:38.562163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.0.205.190:42641)
0808 02:57:38.562380 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:38.562398 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:38.562811 (+ 413us) server_negotiation.cc:407] Connection header received
0808 02:57:38.563115 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:38.563120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:38.563197 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:38.563371 (+ 174us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:57:38.563890 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:38.564640 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:38.566204 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:38.567061 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:38.567218 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:38.567766 (+ 548us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:57:38.582555 (+ 14789us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 02:57:38.583032 (+ 477us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52148: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 02:57:38.583410 8634 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:57:38.562059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:42641 (local address 127.0.0.1:52148)
0808 02:57:38.562692 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:38.562713 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:38.562900 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:38.563384 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:38.563390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:38.563402 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:57:38.563700 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:38.563710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:38.564796 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:38.564803 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:38.566008 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:38.566027 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:38.567256 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:38.567261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:38.567401 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:38.567437 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:57:38.582872 (+ 15435us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 02:57:38.583109 (+ 237us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.205.190:42641: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":368,"threads_started":1}
W20260808 02:57:38.583518 8602 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52148: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38777/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260808 02:57:38.584847 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 8507
I20260808 02:57:38.586354 8593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:38.680276 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 8507
2026-08-08T02:57:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (788 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 02:57:38.742087 822 mini_oidc.cc:152] Starting JWKS server
I20260808 02:57:38.744148 822 webserver.cc:460] Webserver started at https://127.0.0.1:40475/ using document root <none> and password file <none>
I20260808 02:57:38.744375 822 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:57:38.744947 822 webserver.cc:460] Webserver started at http://127.0.0.1:42969/ using document root <none> and password file <none>
2026-08-08T02:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:38Z Disabled control of system clock
I20260808 02:57:38.780212 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:38277
--webserver_interface=127.0.205.190
--webserver_port=0
--builtin_ntp_servers=127.0.205.148:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:38277
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40475/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 02:57:39.125085 8650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:39.125409 8650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:39.125532 8650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:39.132964 8650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:39.133085 8650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:39.133124 8650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:39.133239 8650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:39.133301 8650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:57:39.133440 8650 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:57:39.133498 8650 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40475/jwks.json
I20260808 02:57:39.143013 8650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:38277
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:38277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40475/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:39.144452 8650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:39.146417 8650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:39.159096 8662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:39.159186 8658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:39.159183 8660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:39.160575 8650 server_base.cc:1034] running on GCE node
I20260808 02:57:39.161365 8650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:39.163018 8650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:39.164249 8650 hybrid_clock.cc:648] HybridClock initialized: now 1786157859164176 us; error 60 us; skew 500 ppm
I20260808 02:57:39.164603 8650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:57:39.166988 8650 webserver.cc:460] Webserver started at http://127.0.205.190:42397/ using document root <none> and password file <none>
I20260808 02:57:39.167706 8650 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:39.167801 8650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:39.168090 8650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:39.171211 8650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "b154691083a94ab298f3a0f8552552c2"
format_stamp: "Formatted at 2026-08-08 02:57:39 on dist-test-slave-6ntk"
server_key: "9e92d6aff11749002a69660f92240f49"
server_key_iv: "3e5999901139bb41ed2b135cfb97170a"
server_key_version: "encryptionkey@0"
I20260808 02:57:39.172091 8650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "b154691083a94ab298f3a0f8552552c2"
format_stamp: "Formatted at 2026-08-08 02:57:39 on dist-test-slave-6ntk"
server_key: "9e92d6aff11749002a69660f92240f49"
server_key_iv: "3e5999901139bb41ed2b135cfb97170a"
server_key_version: "encryptionkey@0"
I20260808 02:57:39.177523 8650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260808 02:57:39.181247 8673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:39.182806 8650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:57:39.182999 8650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "b154691083a94ab298f3a0f8552552c2"
format_stamp: "Formatted at 2026-08-08 02:57:39 on dist-test-slave-6ntk"
server_key: "9e92d6aff11749002a69660f92240f49"
server_key_iv: "3e5999901139bb41ed2b135cfb97170a"
server_key_version: "encryptionkey@0"
I20260808 02:57:39.183215 8650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:39.197009 8650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:39.197937 8650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:39.198275 8650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:39.218328 8650 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:38277
I20260808 02:57:39.218335 8738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:38277 every 8 connection(s)
I20260808 02:57:39.219877 8650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:39.223053 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 8650
I20260808 02:57:39.223289 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:39.223932 822 external_mini_cluster.cc:1428] Setting key b4b8fc85db3d632a00434c25b80e2563
I20260808 02:57:39.224136 8739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:39.240183 8743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:39.226050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37894 (local address 127.0.205.190:38277)
0808 02:57:39.227144 (+ 1094us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:39.227170 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:39.227227 (+ 57us) server_negotiation.cc:407] Connection header received
0808 02:57:39.229361 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:39.229402 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:39.230071 (+ 669us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:39.230548 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:39.231128 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:39.233165 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:39.234605 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:39.235700 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:39.235903 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:39.237161 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:39.237173 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:39.237200 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:39.238029 (+ 829us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:39.238136 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:39.238162 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:39.238288 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:39.238495 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 02:57:39.238852 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":349,"threads_started":1}
I20260808 02:57:39.240427 8739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2: Bootstrap starting.
I20260808 02:57:39.243952 8739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:39.244268 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:57:39.245115 8739 log.cc:824] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:39.248461 8739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2: No bootstrap required, opened a new log
I20260808 02:57:39.256703 8739 raft_consensus.cc:348] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b154691083a94ab298f3a0f8552552c2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38277 } }
I20260808 02:57:39.257126 8739 raft_consensus.cc:374] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:39.257239 8739 raft_consensus.cc:729] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b154691083a94ab298f3a0f8552552c2, State: Initialized, Role: FOLLOWER
I20260808 02:57:39.257941 8739 consensus_queue.cc:260] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b154691083a94ab298f3a0f8552552c2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38277 } }
I20260808 02:57:39.258206 8739 raft_consensus.cc:388] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:39.258359 8739 raft_consensus.cc:482] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:39.258540 8739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:39.261122 8739 raft_consensus.cc:504] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b154691083a94ab298f3a0f8552552c2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38277 } }
I20260808 02:57:39.261670 8739 leader_election.cc:302] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b154691083a94ab298f3a0f8552552c2; no voters:
I20260808 02:57:39.262342 8739 leader_election.cc:288] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:39.262526 8745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:39.263266 8745 raft_consensus.cc:686] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [term 1 LEADER]: Becoming Leader. State: Replica: b154691083a94ab298f3a0f8552552c2, State: Running, Role: LEADER
I20260808 02:57:39.263790 8745 consensus_queue.cc:237] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b154691083a94ab298f3a0f8552552c2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38277 } }
I20260808 02:57:39.264642 8739 sys_catalog.cc:564] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:39.267423 8747 sys_catalog.cc:455] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b154691083a94ab298f3a0f8552552c2. Latest consensus state: current_term: 1 leader_uuid: "b154691083a94ab298f3a0f8552552c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b154691083a94ab298f3a0f8552552c2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38277 } } }
I20260808 02:57:39.267951 8747 sys_catalog.cc:458] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:39.268642 8746 sys_catalog.cc:455] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b154691083a94ab298f3a0f8552552c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b154691083a94ab298f3a0f8552552c2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38277 } } }
I20260808 02:57:39.269037 8746 sys_catalog.cc:458] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:39.272128 8757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:39.278195 8757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:39.287374 8757 catalog_manager.cc:1347] Generated new cluster ID: 8cc363eac2ab4c059ff409bcbca70630
I20260808 02:57:39.287505 8757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:39.302569 8757 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:39.304131 8757 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:39.318598 8757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b154691083a94ab298f3a0f8552552c2: Generated new TSK 0
I20260808 02:57:39.319581 8757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:57:39.348904 8743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:39.339979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37902 (local address 127.0.205.190:38277)
0808 02:57:39.340214 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:39.340222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:39.340628 (+ 406us) server_negotiation.cc:407] Connection header received
0808 02:57:39.340874 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:39.340880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:39.340981 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:39.341161 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:57:39.341654 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:39.342854 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:39.344286 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:39.345009 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:39.345143 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:39.345732 (+ 589us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:57:39.348159 (+ 2427us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40475/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 02:57:39.348550 (+ 391us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37902: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40475/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 02:57:39.349076 8743 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37902: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40475/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 02:57:39.349365 8780 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:57:39.339818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:38277 (local address 127.0.0.1:37902)
0808 02:57:39.340508 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:39.340525 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:39.340666 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:39.341179 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:39.341182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:39.341193 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:57:39.341502 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:39.341510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:39.343010 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:39.343014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:39.344133 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:39.344145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:39.345197 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:39.345202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:39.345331 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:39.345365 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:57:39.348387 (+ 3022us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40475/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 02:57:39.348699 (+ 312us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.205.190:38277: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40475/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":357,"threads_started":1}
I20260808 02:57:39.351459 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 8650
I20260808 02:57:39.353005 8732 generic_service.cc:196] Checking for leaks (request via RPC)
E20260808 02:57:39.362123 8781 webserver.cc:533] Webserver: sslize failed:
E20260808 02:57:39.362278 8781 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:57:39.448243 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 8650
2026-08-08T02:57:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (770 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T02:57:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:39Z Disabled control of system clock
I20260808 02:57:39.547369 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:37529
--webserver_interface=127.0.205.190
--webserver_port=0
--builtin_ntp_servers=127.0.205.148:41475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:37529
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/testchainca.cert with env {}
W20260808 02:57:39.895398 8786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:39.895704 8786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:39.895790 8786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:39.902957 8786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:39.903069 8786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:39.903107 8786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:39.903164 8786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:39.903283 8786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:57:39.903345 8786 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:57:39.903386 8786 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 02:57:39.913775 8786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:41475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:37529
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:37529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:39.915194 8786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:39.916507 8786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:39.929530 8792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:39.929615 8797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:39.929570 8793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:39.930644 8786 server_base.cc:1034] running on GCE node
I20260808 02:57:39.931517 8786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:39.933036 8786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:39.934286 8786 hybrid_clock.cc:648] HybridClock initialized: now 1786157859934243 us; error 63 us; skew 500 ppm
I20260808 02:57:39.934695 8786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:57:39.937086 8786 webserver.cc:460] Webserver started at http://127.0.205.190:44199/ using document root <none> and password file <none>
I20260808 02:57:39.937811 8786 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:39.937983 8786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:39.938326 8786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:39.941299 8786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "26c3434c20c743f99db883ac884da9ee"
format_stamp: "Formatted at 2026-08-08 02:57:39 on dist-test-slave-6ntk"
server_key: "9a8325759347a05472b9fad5311d1be2"
server_key_iv: "ec838699bc1726c66828cdf39ace0c13"
server_key_version: "encryptionkey@0"
I20260808 02:57:39.942216 8786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "26c3434c20c743f99db883ac884da9ee"
format_stamp: "Formatted at 2026-08-08 02:57:39 on dist-test-slave-6ntk"
server_key: "9a8325759347a05472b9fad5311d1be2"
server_key_iv: "ec838699bc1726c66828cdf39ace0c13"
server_key_version: "encryptionkey@0"
I20260808 02:57:39.947458 8786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:57:39.951125 8807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:39.952643 8786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:57:39.952843 8786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "26c3434c20c743f99db883ac884da9ee"
format_stamp: "Formatted at 2026-08-08 02:57:39 on dist-test-slave-6ntk"
server_key: "9a8325759347a05472b9fad5311d1be2"
server_key_iv: "ec838699bc1726c66828cdf39ace0c13"
server_key_version: "encryptionkey@0"
I20260808 02:57:39.953068 8786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:39.972728 8786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:39.973639 8786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:39.974016 8786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:39.993062 8786 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:37529
I20260808 02:57:39.993084 8881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:37529 every 8 connection(s)
I20260808 02:57:39.994441 8786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:39.998200 8882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:40.001430 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 8786
I20260808 02:57:40.001642 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:40.002393 822 external_mini_cluster.cc:1428] Setting key b0a90f5fb96d8a7e5893d0ff1b3731c8
I20260808 02:57:40.011158 8882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee: Bootstrap starting.
I20260808 02:57:40.015188 8882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:40.016400 8882 log.cc:824] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:40.016667 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:40.004277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39680 (local address 127.0.205.190:37529)
0808 02:57:40.005280 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:40.005291 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:40.005338 (+ 47us) server_negotiation.cc:407] Connection header received
0808 02:57:40.007046 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:40.007070 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:40.007571 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:40.007925 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:40.008624 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:40.010032 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:40.011825 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:40.012494 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:40.012631 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:40.014120 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:40.014128 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:40.014142 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:57:40.014690 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:40.014799 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:57:40.014820 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:40.014913 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:40.015263 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 02:57:40.015640 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":357,"threads_started":1}
I20260808 02:57:40.019994 8882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee: No bootstrap required, opened a new log
I20260808 02:57:40.020944 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:57:40.028904 8882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c3434c20c743f99db883ac884da9ee" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 37529 } }
I20260808 02:57:40.029337 8882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:40.029466 8882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c3434c20c743f99db883ac884da9ee, State: Initialized, Role: FOLLOWER
I20260808 02:57:40.030164 8882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c3434c20c743f99db883ac884da9ee" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 37529 } }
I20260808 02:57:40.030440 8882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:40.030591 8882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:40.030786 8882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:40.033154 8882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c3434c20c743f99db883ac884da9ee" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 37529 } }
I20260808 02:57:40.033821 8882 leader_election.cc:302] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c3434c20c743f99db883ac884da9ee; no voters:
I20260808 02:57:40.034495 8882 leader_election.cc:288] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:40.034657 8888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:40.035365 8888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [term 1 LEADER]: Becoming Leader. State: Replica: 26c3434c20c743f99db883ac884da9ee, State: Running, Role: LEADER
I20260808 02:57:40.035957 8888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c3434c20c743f99db883ac884da9ee" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 37529 } }
I20260808 02:57:40.036751 8882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:40.039307 8890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26c3434c20c743f99db883ac884da9ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c3434c20c743f99db883ac884da9ee" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 37529 } } }
I20260808 02:57:40.039772 8890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:40.039570 8891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c3434c20c743f99db883ac884da9ee. Latest consensus state: current_term: 1 leader_uuid: "26c3434c20c743f99db883ac884da9ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c3434c20c743f99db883ac884da9ee" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 37529 } } }
I20260808 02:57:40.040853 8891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:40.042228 8895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:40.047639 8895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:40.058041 8895 catalog_manager.cc:1347] Generated new cluster ID: 1bc6381f42e64c3cb37729e3aae66ca7
I20260808 02:57:40.058179 8895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:40.068459 8895 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:40.069844 8895 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:40.083082 8895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26c3434c20c743f99db883ac884da9ee: Generated new TSK 0
I20260808 02:57:40.084056 8895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:57:40.165050 8885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:40.111616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.0.205.190:37529)
0808 02:57:40.111835 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:40.111841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:40.112361 (+ 520us) server_negotiation.cc:407] Connection header received
0808 02:57:40.112655 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:40.112659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:40.112745 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:40.112879 (+ 134us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:57:40.113428 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:40.114295 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:40.116359 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:40.116801 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:40.116896 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:40.117533 (+ 637us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:57:40.164460 (+ 46927us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 02:57:40.164802 (+ 342us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39684: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 02:57:40.165241 8885 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39684: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 02:57:40.165562 8925 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:57:40.111402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:37529 (local address 127.0.0.1:39684)
0808 02:57:40.112239 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:40.112259 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:40.112438 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:40.112934 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:40.112939 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:40.112953 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:57:40.113240 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:40.113251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:40.114457 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:40.114464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:40.116213 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:40.116229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:40.116979 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:40.116987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:40.117194 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:40.117246 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:57:40.164769 (+ 47523us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 02:57:40.165097 (+ 328us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.205.190:37529: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":522,"threads_started":1}
I20260808 02:57:40.167127 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 8786
I20260808 02:57:40.168323 8876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:40.271425 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 8786
2026-08-08T02:57:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (801 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (437 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (446 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:57:41 dist-test-slave-6ntk krb5kdc[8942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:57:41 dist-test-slave-6ntk krb5kdc[8942](info): set up 2 sockets
Aug 08 02:57:41 dist-test-slave-6ntk krb5kdc[8942](info): commencing operation
krb5kdc: starting...
W20260808 02:57:42.492831 822 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.005s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:57:42 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:57:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:57:42.750000 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:38305
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:38305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:43.092161 8958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:43.092502 8958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:43.092656 8958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:43.100101 8958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:43.100255 8958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:43.100368 8958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:43.100476 8958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:43.100543 8958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:43.111035 8958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:38305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:38305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:43.112380 8958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:43.113653 8958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:43.126535 8970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:43.126667 8969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:43.126531 8973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:43.127736 8958 server_base.cc:1034] running on GCE node
I20260808 02:57:43.128608 8958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:43.130147 8958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:43.131399 8958 hybrid_clock.cc:648] HybridClock initialized: now 1786157863131347 us; error 57 us; skew 500 ppm
I20260808 02:57:43.131769 8958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:43 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:43.135175 8958 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:57:43.136698 8958 webserver.cc:460] Webserver started at http://127.0.205.190:34215/ using document root <none> and password file <none>
I20260808 02:57:43.137454 8958 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:43.137611 8958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:43.138036 8958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:43.141294 8958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "1205c5442e15406cbe58eae820e88768"
format_stamp: "Formatted at 2026-08-08 02:57:43 on dist-test-slave-6ntk"
server_key: "b753df37f376d2ef38501e1e789292c7"
server_key_iv: "9621a76fc7c27cc6c42826528d78e4fc"
server_key_version: "encryptionkey@0"
I20260808 02:57:43.142237 8958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "1205c5442e15406cbe58eae820e88768"
format_stamp: "Formatted at 2026-08-08 02:57:43 on dist-test-slave-6ntk"
server_key: "b753df37f376d2ef38501e1e789292c7"
server_key_iv: "9621a76fc7c27cc6c42826528d78e4fc"
server_key_version: "encryptionkey@0"
I20260808 02:57:43.147846 8958 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:57:43.151543 8981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:43.152911 8958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:57:43.153110 8958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "1205c5442e15406cbe58eae820e88768"
format_stamp: "Formatted at 2026-08-08 02:57:43 on dist-test-slave-6ntk"
server_key: "b753df37f376d2ef38501e1e789292c7"
server_key_iv: "9621a76fc7c27cc6c42826528d78e4fc"
server_key_version: "encryptionkey@0"
I20260808 02:57:43.153319 8958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:43.195451 8958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:43.199785 8958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:43.200111 8958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:43.221441 8958 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:38305
I20260808 02:57:43.221465 9047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:38305 every 8 connection(s)
I20260808 02:57:43.223016 8958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:43.225157 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 8958
I20260808 02:57:43.225376 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:43.226152 822 external_mini_cluster.cc:1428] Setting key 9d79f51dd95cf8c5127a343452b8b8ed
I20260808 02:57:43.227289 9049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:43.241811 9049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768: Bootstrap starting.
Aug 08 02:57:43 dist-test-slave-6ntk krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:43.245631 9049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:43.247072 9049 log.cc:824] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:43.250703 9049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768: No bootstrap required, opened a new log
I20260808 02:57:43.252439 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:43.228177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39640 (local address 127.0.205.190:38305)
0808 02:57:43.229197 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:43.229221 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:43.230500 (+ 1279us) server_negotiation.cc:407] Connection header received
0808 02:57:43.232384 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:43.232419 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:43.232938 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:43.233423 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:43.234906 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:43.236811 (+ 1905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:43.238218 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:43.239348 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:43.239530 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:43.243570 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:43.243582 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:43.243604 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:43.243659 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:43.247679 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:43.248632 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:43.248641 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:43.248650 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:43.248886 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:43.249268 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:43.249275 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:43.249278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:43.250004 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:43.250300 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:43.250617 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 02:57:43.251083 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":381,"threads_started":1}
I20260808 02:57:43.259603 9049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1205c5442e15406cbe58eae820e88768" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38305 } }
I20260808 02:57:43.259980 9049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:43.260058 9049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1205c5442e15406cbe58eae820e88768, State: Initialized, Role: FOLLOWER
I20260808 02:57:43.260612 9049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1205c5442e15406cbe58eae820e88768" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38305 } }
I20260808 02:57:43.260825 9049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:43.260924 9049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:43.261053 9049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:43.263207 9049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1205c5442e15406cbe58eae820e88768" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38305 } }
I20260808 02:57:43.263665 9049 leader_election.cc:302] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1205c5442e15406cbe58eae820e88768; no voters:
I20260808 02:57:43.264261 9049 leader_election.cc:288] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:43.264422 9055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:43.265586 9055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [term 1 LEADER]: Becoming Leader. State: Replica: 1205c5442e15406cbe58eae820e88768, State: Running, Role: LEADER
I20260808 02:57:43.266206 9055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1205c5442e15406cbe58eae820e88768" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38305 } }
I20260808 02:57:43.266525 9049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:43.269222 9057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1205c5442e15406cbe58eae820e88768. Latest consensus state: current_term: 1 leader_uuid: "1205c5442e15406cbe58eae820e88768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1205c5442e15406cbe58eae820e88768" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38305 } } }
I20260808 02:57:43.269696 9057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:43.270380 9056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1205c5442e15406cbe58eae820e88768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1205c5442e15406cbe58eae820e88768" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38305 } } }
I20260808 02:57:43.270747 9056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:43.272255 9066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:43.278007 9066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:43.287431 9066 catalog_manager.cc:1347] Generated new cluster ID: 617db42cd81a4adeb099fad3c676111c
I20260808 02:57:43.287559 9066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:43.317145 9066 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:43.318507 9066 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:43.328366 9066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1205c5442e15406cbe58eae820e88768: Generated new TSK 0
I20260808 02:57:43.329255 9066 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:57:43.409448 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38305
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:43.783499 9088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:43.783824 9088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:43.783927 9088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:43.791380 9088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:43.791500 9088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:43.791617 9088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:43.801632 9088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:43.803057 9088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:43.804342 9088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:43.818172 9097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:43.818208 9093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:43.818259 9094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:43.819063 9088 server_base.cc:1034] running on GCE node
I20260808 02:57:43.819703 9088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:43.820716 9088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:43.821974 9088 hybrid_clock.cc:648] HybridClock initialized: now 1786157863821914 us; error 55 us; skew 500 ppm
I20260808 02:57:43.822297 9088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:43 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:43.829365 9088 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:57:43.831075 9088 webserver.cc:460] Webserver started at http://127.0.205.129:43571/ using document root <none> and password file <none>
I20260808 02:57:43.831804 9088 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:43.831971 9088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:43.832288 9088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:43.835414 9088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "a60eea1a72334559af5caa2263a6b713"
format_stamp: "Formatted at 2026-08-08 02:57:43 on dist-test-slave-6ntk"
server_key: "851b9b9fa3e8fac90e1920e188fc1f47"
server_key_iv: "07c11f2a8dac7d9a39489cfb85049e12"
server_key_version: "encryptionkey@0"
I20260808 02:57:43.836268 9088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "a60eea1a72334559af5caa2263a6b713"
format_stamp: "Formatted at 2026-08-08 02:57:43 on dist-test-slave-6ntk"
server_key: "851b9b9fa3e8fac90e1920e188fc1f47"
server_key_iv: "07c11f2a8dac7d9a39489cfb85049e12"
server_key_version: "encryptionkey@0"
I20260808 02:57:43.841559 9088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:57:43.845130 9108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:43.846580 9088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:43.846765 9088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "a60eea1a72334559af5caa2263a6b713"
format_stamp: "Formatted at 2026-08-08 02:57:43 on dist-test-slave-6ntk"
server_key: "851b9b9fa3e8fac90e1920e188fc1f47"
server_key_iv: "07c11f2a8dac7d9a39489cfb85049e12"
server_key_version: "encryptionkey@0"
I20260808 02:57:43.846994 9088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:43.860013 9088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:43.864212 9088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:43.864526 9088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:43.865514 9088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:43.866995 9088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:43.867110 9088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:43.867259 9088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:43.867331 9088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:43.901795 9088 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:46241
I20260808 02:57:43.901851 9258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:46241 every 8 connection(s)
I20260808 02:57:43.903201 9088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:43.908713 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 9088
I20260808 02:57:43.908953 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:57:43.909772 822 external_mini_cluster.cc:1428] Setting key af31b1b589c2d0e324330acba2d6356d
Aug 08 02:57:43 dist-test-slave-6ntk krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:43.925351 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:43.905947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:46339 (local address 127.0.205.190:38305)
0808 02:57:43.906430 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:43.906434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:43.907433 (+ 999us) server_negotiation.cc:407] Connection header received
0808 02:57:43.910031 (+ 2598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:43.910036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:43.910111 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:43.910210 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:43.911933 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:43.912664 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:43.914375 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:43.914788 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:43.914948 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:43.918677 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:43.918682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:43.918685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:43.918719 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:43.922059 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:43.922837 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:43.922843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:43.922846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:43.922980 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:43.923459 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:43.923465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:43.923473 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:43.923750 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:43.923918 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:43.924726 (+ 808us) server_negotiation.cc:299] Negotiation successful
0808 02:57:43.924961 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":224,"threads_started":1}
I20260808 02:57:43.926446 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:43.906500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:38305 (local address 127.0.205.129:46339)
0808 02:57:43.907250 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:43.907292 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:43.909662 (+ 2370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:43.910553 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:43.910564 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:43.911114 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:43.911735 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:43.911754 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:43.912851 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:43.912855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:43.914213 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:43.914231 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:43.915050 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:43.915060 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:43.915314 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:43.916063 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:43.916088 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:43.918437 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:43.922212 (+ 3775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:43.922222 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:43.922240 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:43.922697 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:43.923102 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:43.923107 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:43.923111 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:43.923321 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:43.923889 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:43.923907 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:43.924315 (+ 408us) client_negotiation.cc:769] Sending connection context
0808 02:57:43.924594 (+ 279us) client_negotiation.cc:240] Negotiation successful
0808 02:57:43.925169 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":307,"threads_started":1}
I20260808 02:57:43.928212 9260 heartbeater.cc:347] Connected to a master server at 127.0.205.190:38305
I20260808 02:57:43.928597 9260 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:43.929569 9260 heartbeater.cc:511] Master 127.0.205.190:38305 requested a full tablet report, sending...
I20260808 02:57:43.932317 9007 ts_manager.cc:194] Registered new tserver with Master: a60eea1a72334559af5caa2263a6b713 (127.0.205.129:46241)
I20260808 02:57:43.934616 9007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:46339
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:57:43.968948 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38305
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:44.330328 9267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:44.330610 9267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:44.330811 9267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:44.338241 9267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:44.338357 9267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:44.338567 9267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:44.348625 9267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:44.350024 9267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:44.351516 9267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:44.365454 9275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:44.365423 9279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:44.365492 9276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:44.366217 9267 server_base.cc:1034] running on GCE node
I20260808 02:57:44.366923 9267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:44.367992 9267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:44.369256 9267 hybrid_clock.cc:648] HybridClock initialized: now 1786157864369187 us; error 60 us; skew 500 ppm
I20260808 02:57:44.369690 9267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:44 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:44.373157 9267 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:57:44.374943 9267 webserver.cc:460] Webserver started at http://127.0.205.130:37517/ using document root <none> and password file <none>
I20260808 02:57:44.375720 9267 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:44.375859 9267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:44.376192 9267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:44.379426 9267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "2575ae161ce74a65958b4355d1e511d9"
format_stamp: "Formatted at 2026-08-08 02:57:44 on dist-test-slave-6ntk"
server_key: "8031129eb80ea2cf03bc2b0c84ddc621"
server_key_iv: "c47a3506c8d644d4fe4889a6bda7529f"
server_key_version: "encryptionkey@0"
I20260808 02:57:44.380264 9267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "2575ae161ce74a65958b4355d1e511d9"
format_stamp: "Formatted at 2026-08-08 02:57:44 on dist-test-slave-6ntk"
server_key: "8031129eb80ea2cf03bc2b0c84ddc621"
server_key_iv: "c47a3506c8d644d4fe4889a6bda7529f"
server_key_version: "encryptionkey@0"
I20260808 02:57:44.385618 9267 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:57:44.389127 9287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:44.390614 9267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:44.390803 9267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "2575ae161ce74a65958b4355d1e511d9"
format_stamp: "Formatted at 2026-08-08 02:57:44 on dist-test-slave-6ntk"
server_key: "8031129eb80ea2cf03bc2b0c84ddc621"
server_key_iv: "c47a3506c8d644d4fe4889a6bda7529f"
server_key_version: "encryptionkey@0"
I20260808 02:57:44.391018 9267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:44.406003 9267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:44.410136 9267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:44.410457 9267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:44.411451 9267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:44.412851 9267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:44.412958 9267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:44.413125 9267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:44.413203 9267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:44.447252 9267 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:38653
I20260808 02:57:44.447302 9447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:38653 every 8 connection(s)
I20260808 02:57:44.448642 9267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:44.457065 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 9267
I20260808 02:57:44.457288 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:57:44.457737 822 external_mini_cluster.cc:1428] Setting key aa1b38b4922488e529960126aef7ec0b
Aug 08 02:57:44 dist-test-slave-6ntk krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:44.470471 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:44.451550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:47027 (local address 127.0.205.190:38305)
0808 02:57:44.452100 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:44.452105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:44.453009 (+ 904us) server_negotiation.cc:407] Connection header received
0808 02:57:44.454967 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:44.454973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:44.455066 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:44.455191 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:44.456742 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:44.457676 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:44.459520 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:44.459973 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:44.460114 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:44.463842 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:44.463848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:44.463851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:44.463883 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:44.466778 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:44.467659 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:44.467667 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:44.467673 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:44.467873 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:44.468497 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:44.468501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:44.468503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:44.468772 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:44.468894 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:44.469983 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0808 02:57:44.470172 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":294,"threads_started":1}
I20260808 02:57:44.471998 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:44.452109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:38305 (local address 127.0.205.130:47027)
0808 02:57:44.452844 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:44.452892 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:44.454684 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:44.455439 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:44.455447 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:44.455971 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:44.456548 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:44.456565 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:44.457854 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:44.457858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:44.459362 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:44.459375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:44.460221 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:44.460228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:44.460459 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:44.461160 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:44.461183 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:44.463589 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:44.466922 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:44.466932 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:44.466950 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:44.467480 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:44.467992 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:44.467995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:44.467997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:44.468363 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:44.468903 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:44.468913 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:44.469479 (+ 566us) client_negotiation.cc:769] Sending connection context
0808 02:57:44.469810 (+ 331us) client_negotiation.cc:240] Negotiation successful
0808 02:57:44.470456 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":292,"threads_started":1}
I20260808 02:57:44.473730 9448 heartbeater.cc:347] Connected to a master server at 127.0.205.190:38305
I20260808 02:57:44.474097 9448 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:44.474905 9448 heartbeater.cc:511] Master 127.0.205.190:38305 requested a full tablet report, sending...
I20260808 02:57:44.476545 9007 ts_manager.cc:194] Registered new tserver with Master: 2575ae161ce74a65958b4355d1e511d9 (127.0.205.130:38653)
I20260808 02:57:44.477411 9007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:47027
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:57:44.515451 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38305
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:44.876329 9454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:44.876695 9454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:44.876911 9454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:44.884677 9454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:44.884863 9454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:44.885064 9454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:44.896040 9454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:44.897480 9454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:44.899034 9454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:44.915038 9461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:44.915083 9462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:44.915143 9464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:44.916034 9454 server_base.cc:1034] running on GCE node
I20260808 02:57:44.916756 9454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:44.917850 9454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:44.919178 9454 hybrid_clock.cc:648] HybridClock initialized: now 1786157864919118 us; error 47 us; skew 500 ppm
I20260808 02:57:44.919646 9454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:44 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:44.923106 9454 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:57:44.924747 9454 webserver.cc:460] Webserver started at http://127.0.205.131:41221/ using document root <none> and password file <none>
I20260808 02:57:44.925490 9454 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:44.925652 9454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:44.926074 9454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:44.929116 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "cfa491847be3448bb66043c1c3add9a8"
format_stamp: "Formatted at 2026-08-08 02:57:44 on dist-test-slave-6ntk"
server_key: "57c6f1a682b5d8513a84f43dac619515"
server_key_iv: "67500dd92f5c27e18dfe12671fc0bac9"
server_key_version: "encryptionkey@0"
I20260808 02:57:44.930109 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "cfa491847be3448bb66043c1c3add9a8"
format_stamp: "Formatted at 2026-08-08 02:57:44 on dist-test-slave-6ntk"
server_key: "57c6f1a682b5d8513a84f43dac619515"
server_key_iv: "67500dd92f5c27e18dfe12671fc0bac9"
server_key_version: "encryptionkey@0"
I20260808 02:57:44.935478 9454 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:57:44.937898 9260 heartbeater.cc:503] Master 127.0.205.190:38305 was elected leader, sending a full tablet report...
I20260808 02:57:44.939464 9473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:44.941073 9454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:57:44.941268 9454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "cfa491847be3448bb66043c1c3add9a8"
format_stamp: "Formatted at 2026-08-08 02:57:44 on dist-test-slave-6ntk"
server_key: "57c6f1a682b5d8513a84f43dac619515"
server_key_iv: "67500dd92f5c27e18dfe12671fc0bac9"
server_key_version: "encryptionkey@0"
I20260808 02:57:44.941500 9454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:44.969550 9454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:44.974535 9454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:44.974798 9454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:44.975797 9454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:44.977195 9454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:44.977272 9454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:44.977356 9454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:44.977409 9454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:45.012506 9454 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:41161
I20260808 02:57:45.012552 9623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:41161 every 8 connection(s)
I20260808 02:57:45.013831 9454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:57:45.021736 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 9454
I20260808 02:57:45.021952 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:57:45.022435 822 external_mini_cluster.cc:1428] Setting key 7decdb8ca89ff27b10aede17864bbf3f
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:57:45.033928 9628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:45.016624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:43055 (local address 127.0.205.190:38305)
0808 02:57:45.017122 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:45.017127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:45.018095 (+ 968us) server_negotiation.cc:407] Connection header received
0808 02:57:45.019942 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:45.019947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:45.020037 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:45.020161 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:45.021705 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.022677 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.024256 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.024700 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.024855 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:45.028433 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:45.028438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:45.028442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:45.028476 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:45.031087 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.031849 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.031853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.031856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.031984 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.032364 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.032367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.032369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.032632 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:45.032750 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:45.033413 (+ 663us) server_negotiation.cc:299] Negotiation successful
0808 02:57:45.033580 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":266,"threads_started":1}
I20260808 02:57:45.034927 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:45.017143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:38305 (local address 127.0.205.131:43055)
0808 02:57:45.017923 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:45.017984 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:45.019644 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:45.020409 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:45.020417 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:45.020966 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:45.021527 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:45.021544 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.022830 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.022840 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:45.024082 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:45.024094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.024945 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.024953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:45.025216 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:45.025953 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:45.025981 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:45.028207 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:45.031228 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:45.031235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:45.031248 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:45.031736 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:45.032083 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:45.032086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:45.032088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:45.032262 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:45.032738 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:45.032744 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:45.033074 (+ 330us) client_negotiation.cc:769] Sending connection context
0808 02:57:45.033316 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 02:57:45.033734 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":290,"threads_started":1}
I20260808 02:57:45.036484 9625 heartbeater.cc:347] Connected to a master server at 127.0.205.190:38305
I20260808 02:57:45.036810 9625 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:45.037603 9625 heartbeater.cc:511] Master 127.0.205.190:38305 requested a full tablet report, sending...
I20260808 02:57:45.039172 9007 ts_manager.cc:194] Registered new tserver with Master: cfa491847be3448bb66043c1c3add9a8 (127.0.205.131:41161)
I20260808 02:57:45.039973 9007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:43055
I20260808 02:57:45.050144 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/test-admin.keytab.
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:45.085108 822 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 02:57:45.091336 9635 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:57:45.088311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51648 (local address 127.0.205.129:46241)
0808 02:57:45.089060 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:45.089067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:45.089093 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:57:45.089154 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:45.089160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:45.089385 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:45.089583 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:45.090841 (+ 1258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51648: BlockingRecv error: recv got EOF from 127.0.0.1:51648 (error 108)
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":346,"threads_started":1}
W20260808 02:57:45.091497 822 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:45.093349 822 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:45.094981 822 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:45.112033 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:45.096964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.0.205.129:46241)
0808 02:57:45.097191 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:45.097197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:45.097389 (+ 192us) server_negotiation.cc:407] Connection header received
0808 02:57:45.097613 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:45.097625 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:45.097689 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:45.097801 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:45.098895 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.099919 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.101179 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.101771 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.101966 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:45.105706 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:45.105718 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:45.105735 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:45.105777 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:45.109190 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.110069 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.110076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.110078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.110210 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.110578 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.110581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.110583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.110960 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:45.111182 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:45.111458 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 02:57:45.111745 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 02:57:45.118269 9635 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:57:45.116339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51664 (local address 127.0.205.129:46241)
0808 02:57:45.116572 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:45.116578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:45.116786 (+ 208us) server_negotiation.cc:407] Connection header received
0808 02:57:45.116980 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:45.116990 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:45.117060 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:45.117188 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:45.118026 (+ 838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51664: BlockingRecv error: recv got EOF from 127.0.0.1:51664 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 02:57:45.118463 822 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:45.120148 822 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:45.121500 822 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:45.136205 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:45.122892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51666 (local address 127.0.205.129:46241)
0808 02:57:45.123057 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:45.123062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:45.123534 (+ 472us) server_negotiation.cc:407] Connection header received
0808 02:57:45.123672 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:45.123676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:45.123733 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:45.123815 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:45.124848 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.125632 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.126967 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.127394 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.127549 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:45.131208 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:45.131213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:45.131216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:45.131247 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:45.133560 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.134300 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.134307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.134310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.134448 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.134839 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.134846 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.134853 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.135126 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:45.135415 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:45.135711 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 02:57:45.135931 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 02:57:45.140878 9635 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:57:45.139073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51672 (local address 127.0.205.129:46241)
0808 02:57:45.139275 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:45.139282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:45.139635 (+ 353us) server_negotiation.cc:407] Connection header received
0808 02:57:45.139737 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:45.139740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:45.139807 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:45.139908 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:45.140622 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51672: BlockingRecv error: recv got EOF from 127.0.0.1:51672 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 02:57:45.141054 822 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786157811617362-822-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:45.142653 822 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:45.143971 822 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:57:45.158775 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:45.145239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51688 (local address 127.0.205.129:46241)
0808 02:57:45.145438 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:45.145445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:45.145729 (+ 284us) server_negotiation.cc:407] Connection header received
0808 02:57:45.146002 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:45.146008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:45.146082 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:45.146189 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:45.147115 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.147831 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.149092 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.149567 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.149711 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:45.153460 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:45.153466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:45.153472 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:45.153511 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:45.156107 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.156781 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.156788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.156791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.156952 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.157351 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.157358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.157365 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.157640 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:45.157985 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:45.158243 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 02:57:45.158461 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 02:57:45.160389 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 9088
I20260808 02:57:45.172984 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:45.161138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51692 (local address 127.0.205.129:46241)
0808 02:57:45.161319 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:45.161327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:45.161618 (+ 291us) server_negotiation.cc:407] Connection header received
0808 02:57:45.161800 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:45.161806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:45.161914 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:45.162023 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:45.162973 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.163823 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.165031 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.165508 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.165661 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:45.167859 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:45.167865 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:45.167870 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:45.167910 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:45.170375 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.171025 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.171032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.171035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.171183 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.171568 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.171574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.171579 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.171846 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:45.172115 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:45.172448 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 02:57:45.172647 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:57:45.173604 9253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:45.264163 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 9088
I20260808 02:57:45.287773 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 9267
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:57:45.304311 9650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:45.288696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42142 (local address 127.0.205.130:38653)
0808 02:57:45.289283 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:45.289290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:45.289327 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:57:45.289394 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:45.289402 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:45.289619 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:45.289758 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:45.290879 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.291906 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.293186 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.293794 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.294116 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:45.298262 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:45.298274 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:45.298289 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:45.298321 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:45.301181 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.302030 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.302038 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.302040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.302167 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.302588 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.302595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.302599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.302999 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:45.303354 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:45.303705 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 02:57:45.303983 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":310,"threads_started":1}
I20260808 02:57:45.305617 9443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:45.390053 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 9267
I20260808 02:57:45.415387 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 9454
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[8942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:57:45.432384 9653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:45.416466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.0.205.131:41161)
0808 02:57:45.417067 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:45.417075 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:45.417104 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:57:45.417168 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:45.417175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:45.417398 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:45.417534 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:45.418712 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.419765 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.420987 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:45.421565 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:45.421664 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:45.426090 (+ 4426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:45.426104 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:45.426121 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:45.426164 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:45.429224 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.430124 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.430133 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.430135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.430286 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:45.430711 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:45.430715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:45.430717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:45.431107 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:45.431394 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:45.431581 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 02:57:45.431838 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":270,"threads_started":1}
I20260808 02:57:45.433818 9616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:45.518594 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 9454
I20260808 02:57:45.543879 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 8958
I20260808 02:57:45.545209 9042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:45.650188 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 8958
2026-08-08T02:57:45Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4502 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[9660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[9660](info): set up 2 sockets
Aug 08 02:57:45 dist-test-slave-6ntk krb5kdc[9660](info): commencing operation
krb5kdc: starting...
W20260808 02:57:46.961272 822 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:57:47 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:57:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.190.keytab.
Entry for principal oryx/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.190.keytab.
I20260808 02:57:47.227243 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:34241
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.190.keytab
--principal=oryx/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:34241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:47.577561 9680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:47.577919 9680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:47.578097 9680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:47.585474 9680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:47.585599 9680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:47.585709 9680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:47.585820 9680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:47.585894 9680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:47.596298 9680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:34241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.190.keytab
--principal=oryx/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:34241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:47.597723 9680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:47.599275 9680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:47.611980 9690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:47.612020 9692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:47.612011 9689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:47.613019 9680 server_base.cc:1034] running on GCE node
I20260808 02:57:47.614012 9680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:47.615533 9680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:47.616816 9680 hybrid_clock.cc:648] HybridClock initialized: now 1786157867616759 us; error 46 us; skew 500 ppm
I20260808 02:57:47.617268 9680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:47 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157867, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:47.620733 9680 init.cc:377] Logged in from keytab as oryx/127.0.205.190@KRBTEST.COM (short username oryx)
I20260808 02:57:47.622489 9680 webserver.cc:460] Webserver started at http://127.0.205.190:43451/ using document root <none> and password file <none>
I20260808 02:57:47.623198 9680 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:47.623303 9680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:47.623579 9680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:47.626820 9680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "55a2aa7994d1497196b00b86336a13a1"
format_stamp: "Formatted at 2026-08-08 02:57:47 on dist-test-slave-6ntk"
server_key: "0d7c675d45dc7733d9a2fb290a14c7a2"
server_key_iv: "d4e5a435d8101fec0f5ca6c7706bb123"
server_key_version: "encryptionkey@0"
I20260808 02:57:47.627559 9680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "55a2aa7994d1497196b00b86336a13a1"
format_stamp: "Formatted at 2026-08-08 02:57:47 on dist-test-slave-6ntk"
server_key: "0d7c675d45dc7733d9a2fb290a14c7a2"
server_key_iv: "d4e5a435d8101fec0f5ca6c7706bb123"
server_key_version: "encryptionkey@0"
I20260808 02:57:47.633116 9680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 02:57:47.636883 9700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:47.638516 9680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:47.638718 9680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "55a2aa7994d1497196b00b86336a13a1"
format_stamp: "Formatted at 2026-08-08 02:57:47 on dist-test-slave-6ntk"
server_key: "0d7c675d45dc7733d9a2fb290a14c7a2"
server_key_iv: "d4e5a435d8101fec0f5ca6c7706bb123"
server_key_version: "encryptionkey@0"
I20260808 02:57:47.638950 9680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:47.655736 9680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:47.660557 9680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:47.660892 9680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:47.680832 9680 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:34241
I20260808 02:57:47.680876 9767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:34241 every 8 connection(s)
I20260808 02:57:47.682541 9680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:47.683630 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 9680
I20260808 02:57:47.683909 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:47.684597 822 external_mini_cluster.cc:1428] Setting key 27564d776ff65d19f388d103203eed88
I20260808 02:57:47.687727 9768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 02:57:47 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:57:47.701385 9768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1: Bootstrap starting.
I20260808 02:57:47.704761 9768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:47.706039 9768 log.cc:824] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:47.709538 9768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1: No bootstrap required, opened a new log
I20260808 02:57:47.710183 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:47.687009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46804 (local address 127.0.205.190:34241)
0808 02:57:47.687876 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:47.687893 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:47.687947 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:57:47.690376 (+ 2429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:47.690421 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:47.690953 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:47.691477 (+ 524us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:47.693017 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:47.695141 (+ 2124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:47.696524 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:47.697628 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:47.697829 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:47.702268 (+ 4439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:47.702276 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:47.702291 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:47.702343 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:47.705582 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:47.706513 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:47.706520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:47.706526 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:47.706792 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:47.707222 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:47.707226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:47.707229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:47.707856 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:47.708105 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:47.708587 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 02:57:47.708994 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":339,"threads_started":1}
I20260808 02:57:47.717974 9768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } }
I20260808 02:57:47.718382 9768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:47.718508 9768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55a2aa7994d1497196b00b86336a13a1, State: Initialized, Role: FOLLOWER
I20260808 02:57:47.719229 9768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } }
I20260808 02:57:47.719545 9768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:47.719707 9768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:47.719940 9768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:47.722235 9768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } }
I20260808 02:57:47.722709 9768 leader_election.cc:302] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55a2aa7994d1497196b00b86336a13a1; no voters:
I20260808 02:57:47.723277 9768 leader_election.cc:288] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:47.723419 9773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:47.724143 9773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 1 LEADER]: Becoming Leader. State: Replica: 55a2aa7994d1497196b00b86336a13a1, State: Running, Role: LEADER
I20260808 02:57:47.724787 9773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } }
I20260808 02:57:47.725540 9768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:47.728164 9775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } }
I20260808 02:57:47.728600 9775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:47.728979 9776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a2aa7994d1497196b00b86336a13a1. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } }
I20260808 02:57:47.729481 9776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:47.735093 9784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:47.739950 9784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:47.750231 9784 catalog_manager.cc:1347] Generated new cluster ID: 12460217a9d5465098796d959476dd8e
I20260808 02:57:47.750355 9784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:47.787886 9784 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:47.789354 9784 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:47.795882 9784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1: Generated new TSK 0
I20260808 02:57:47.797035 9784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab.
Entry for principal oryx/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab.
I20260808 02:57:47.870421 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:34241
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab
--principal=oryx/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:48.224069 9800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:48.224526 9800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:48.224699 9800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:48.233141 9800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:48.233256 9800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:48.233453 9800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:48.244118 9800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab
--principal=oryx/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:34241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:48.245554 9800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:48.247022 9800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:48.260880 9806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:48.260938 9810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:48.260910 9807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:48.261819 9800 server_base.cc:1034] running on GCE node
I20260808 02:57:48.262594 9800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:48.263654 9800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:48.264873 9800 hybrid_clock.cc:648] HybridClock initialized: now 1786157868264819 us; error 49 us; skew 500 ppm
I20260808 02:57:48.265225 9800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:48 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157868, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:48.268676 9800 init.cc:377] Logged in from keytab as oryx/127.0.205.129@KRBTEST.COM (short username oryx)
I20260808 02:57:48.270342 9800 webserver.cc:460] Webserver started at http://127.0.205.129:45709/ using document root <none> and password file <none>
I20260808 02:57:48.271174 9800 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:48.271317 9800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:48.271653 9800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:48.274704 9800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "17cab275b91140e5bbcea6f6bd266e07"
format_stamp: "Formatted at 2026-08-08 02:57:48 on dist-test-slave-6ntk"
server_key: "cd7bebb928a722e27381690d005cbbc2"
server_key_iv: "97f65b8fa41e789553ef2e4327a5099f"
server_key_version: "encryptionkey@0"
I20260808 02:57:48.275518 9800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "17cab275b91140e5bbcea6f6bd266e07"
format_stamp: "Formatted at 2026-08-08 02:57:48 on dist-test-slave-6ntk"
server_key: "cd7bebb928a722e27381690d005cbbc2"
server_key_iv: "97f65b8fa41e789553ef2e4327a5099f"
server_key_version: "encryptionkey@0"
I20260808 02:57:48.280757 9800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 02:57:48.284327 9821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:48.285727 9800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:57:48.286007 9800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "17cab275b91140e5bbcea6f6bd266e07"
format_stamp: "Formatted at 2026-08-08 02:57:48 on dist-test-slave-6ntk"
server_key: "cd7bebb928a722e27381690d005cbbc2"
server_key_iv: "97f65b8fa41e789553ef2e4327a5099f"
server_key_version: "encryptionkey@0"
I20260808 02:57:48.286248 9800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:48.311321 9800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:48.315542 9800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:48.315860 9800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:48.316924 9800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:48.318375 9800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:48.318500 9800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:48.318656 9800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:48.318735 9800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:48.353538 9800 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:46555
I20260808 02:57:48.353590 9964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:46555 every 8 connection(s)
I20260808 02:57:48.354979 9800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:48.357707 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 9800
I20260808 02:57:48.357978 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:57:48.358755 822 external_mini_cluster.cc:1428] Setting key e751c193028d08c859ab43272a7691e8
Aug 08 02:57:48 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157868, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:57:48.378178 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:48.357695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:37837 (local address 127.0.205.190:34241)
0808 02:57:48.358272 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:48.358279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:48.359621 (+ 1342us) server_negotiation.cc:407] Connection header received
0808 02:57:48.361596 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:48.361605 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:48.361698 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:48.361819 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:48.363435 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:48.364555 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:48.366205 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:48.366913 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:48.367061 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:48.370869 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:48.370877 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:48.370882 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:48.370929 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:48.374670 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:48.375436 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:48.375444 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:48.375450 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:48.375609 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:48.376047 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:48.376054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:48.376061 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:48.376487 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:48.376646 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:48.377357 (+ 711us) server_negotiation.cc:299] Negotiation successful
0808 02:57:48.377650 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":300,"threads_started":1}
I20260808 02:57:48.378860 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:48.358304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.129:37837)
0808 02:57:48.359198 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:48.359248 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:48.361299 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:48.362105 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:48.362114 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:48.362654 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:48.363244 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:48.363261 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:48.364713 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:48.364717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:48.366051 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:48.366063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:48.367131 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:48.367135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:48.367328 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:48.368067 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:48.368089 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:48.370606 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:48.374829 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:48.374836 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:48.374850 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:48.375280 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:48.375720 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:48.375723 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:48.375725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:48.375905 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:48.376620 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:48.376625 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:48.376941 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 02:57:48.377181 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 02:57:48.377580 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":347,"threads_started":1}
I20260808 02:57:48.380604 9967 heartbeater.cc:347] Connected to a master server at 127.0.205.190:34241
I20260808 02:57:48.380936 9967 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:48.381770 9967 heartbeater.cc:511] Master 127.0.205.190:34241 requested a full tablet report, sending...
I20260808 02:57:48.384311 9722 ts_manager.cc:194] Registered new tserver with Master: 17cab275b91140e5bbcea6f6bd266e07 (127.0.205.129:46555)
WARNING: no policy specified for oryx/127.0.205.130@KRBTEST.COM; defaulting to no policy
I20260808 02:57:48.386662 9722 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:37837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab.
Entry for principal oryx/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab.
I20260808 02:57:48.422101 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:34241
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab
--principal=oryx/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:48.774778 9978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:48.775081 9978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:48.775175 9978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:48.782616 9978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:48.782732 9978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:48.782851 9978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:48.793023 9978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab
--principal=oryx/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:34241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:48.794430 9978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:48.795701 9978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:48.809441 9985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:48.809489 9988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:48.809422 9986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:48.810470 9978 server_base.cc:1034] running on GCE node
I20260808 02:57:48.811035 9978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:48.811983 9978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:48.813221 9978 hybrid_clock.cc:648] HybridClock initialized: now 1786157868813161 us; error 43 us; skew 500 ppm
I20260808 02:57:48.813572 9978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:48 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157868, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:48.816880 9978 init.cc:377] Logged in from keytab as oryx/127.0.205.130@KRBTEST.COM (short username oryx)
I20260808 02:57:48.818542 9978 webserver.cc:460] Webserver started at http://127.0.205.130:43125/ using document root <none> and password file <none>
I20260808 02:57:48.819259 9978 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:48.819425 9978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:48.819752 9978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:48.822793 9978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "be391462969e4dad920ea4aa06ee1169"
format_stamp: "Formatted at 2026-08-08 02:57:48 on dist-test-slave-6ntk"
server_key: "d93506c411e4a43447e33c6864714c0b"
server_key_iv: "67688e942e6faa612e285ba8921321ee"
server_key_version: "encryptionkey@0"
I20260808 02:57:48.823624 9978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "be391462969e4dad920ea4aa06ee1169"
format_stamp: "Formatted at 2026-08-08 02:57:48 on dist-test-slave-6ntk"
server_key: "d93506c411e4a43447e33c6864714c0b"
server_key_iv: "67688e942e6faa612e285ba8921321ee"
server_key_version: "encryptionkey@0"
I20260808 02:57:48.829291 9978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 02:57:48.832825 9997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:48.834311 9978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:48.834509 9978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "be391462969e4dad920ea4aa06ee1169"
format_stamp: "Formatted at 2026-08-08 02:57:48 on dist-test-slave-6ntk"
server_key: "d93506c411e4a43447e33c6864714c0b"
server_key_iv: "67688e942e6faa612e285ba8921321ee"
server_key_version: "encryptionkey@0"
I20260808 02:57:48.834722 9978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:48.873133 9978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:48.877490 9978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:48.877825 9978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:48.879056 9978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:48.880473 9978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:48.880589 9978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:48.880759 9978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:48.880838 9978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:48.917143 9978 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:33955
I20260808 02:57:48.917243 10139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:33955 every 8 connection(s)
I20260808 02:57:48.918561 9978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:48.918954 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 9978
I20260808 02:57:48.919173 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:57:48.919914 822 external_mini_cluster.cc:1428] Setting key f31f2cee3bce8e1e6dc916424e5b6621
Aug 08 02:57:48 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157868, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:57:48.944188 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:48.921436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:56863 (local address 127.0.205.190:34241)
0808 02:57:48.921975 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:48.921982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:48.923310 (+ 1328us) server_negotiation.cc:407] Connection header received
0808 02:57:48.926444 (+ 3134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:48.926451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:48.926538 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:48.926631 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:48.928944 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:48.929819 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:48.932404 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:48.932837 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:48.932988 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:48.937446 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:48.937451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:48.937455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:48.937490 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:48.940434 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:48.941461 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:48.941465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:48.941468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:48.941598 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:48.942175 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:48.942179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:48.942181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:48.942472 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:48.942586 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:48.943600 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0808 02:57:48.943778 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":268,"threads_started":1}
I20260808 02:57:48.945823 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:48.922202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.130:56863)
0808 02:57:48.923101 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:48.923160 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:48.926040 (+ 2880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:48.926954 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:48.926968 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:48.927804 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:48.928718 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:48.928748 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:48.930016 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:48.930025 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:48.932231 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:48.932245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:48.933105 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:48.933112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:48.933362 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:48.934132 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:48.934161 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:48.937133 (+ 2972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:48.940592 (+ 3459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:48.940604 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:48.940630 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:48.941325 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:48.941723 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:48.941730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:48.941736 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:48.942061 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:48.942619 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:48.942629 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:48.943154 (+ 525us) client_negotiation.cc:769] Sending connection context
0808 02:57:48.943502 (+ 348us) client_negotiation.cc:240] Negotiation successful
0808 02:57:48.944152 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":325,"threads_started":1}
WARNING: no policy specified for oryx/127.0.205.131@KRBTEST.COM; defaulting to no policy
I20260808 02:57:48.947562 10140 heartbeater.cc:347] Connected to a master server at 127.0.205.190:34241
I20260808 02:57:48.947894 10140 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:48.948725 10140 heartbeater.cc:511] Master 127.0.205.190:34241 requested a full tablet report, sending...
I20260808 02:57:48.950428 9722 ts_manager.cc:194] Registered new tserver with Master: be391462969e4dad920ea4aa06ee1169 (127.0.205.130:33955)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.131@KRBTEST.COM" created.
I20260808 02:57:48.951377 9722 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:56863
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab.
Entry for principal oryx/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab.
I20260808 02:57:48.981673 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:34241
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab
--principal=oryx/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:49.339890 10150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:49.340170 10150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:49.340339 10150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:49.347860 10150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:49.347985 10150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:49.348177 10150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:49.358562 10150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab
--principal=oryx/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:34241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:49.359853 10150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:49.361164 10150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:49.375401 10159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:49.375468 10162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:49.375430 10160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:49.376183 10150 server_base.cc:1034] running on GCE node
I20260808 02:57:49.376819 10150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:49.377842 10150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:49.379156 10150 hybrid_clock.cc:648] HybridClock initialized: now 1786157869379081 us; error 73 us; skew 500 ppm
I20260808 02:57:49.379485 10150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:49 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157869, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:49.382948 10150 init.cc:377] Logged in from keytab as oryx/127.0.205.131@KRBTEST.COM (short username oryx)
I20260808 02:57:49.384507 10150 webserver.cc:460] Webserver started at http://127.0.205.131:35245/ using document root <none> and password file <none>
I20260808 02:57:49.385233 10150 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:49.385404 10150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:49.385742 10150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:49.388939 10150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "e0e98ea59c2c4ec694d2b1c5899998d1"
format_stamp: "Formatted at 2026-08-08 02:57:49 on dist-test-slave-6ntk"
server_key: "a51a1933aa6955b6436e13b4798653bf"
server_key_iv: "cc8a0c47a86d89a1a436628f426812bd"
server_key_version: "encryptionkey@0"
I20260808 02:57:49.390018 9967 heartbeater.cc:503] Master 127.0.205.190:34241 was elected leader, sending a full tablet report...
I20260808 02:57:49.390283 10150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "e0e98ea59c2c4ec694d2b1c5899998d1"
format_stamp: "Formatted at 2026-08-08 02:57:49 on dist-test-slave-6ntk"
server_key: "a51a1933aa6955b6436e13b4798653bf"
server_key_iv: "cc8a0c47a86d89a1a436628f426812bd"
server_key_version: "encryptionkey@0"
I20260808 02:57:49.396358 10150 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 02:57:49.400276 10169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:49.401805 10150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:57:49.402038 10150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "e0e98ea59c2c4ec694d2b1c5899998d1"
format_stamp: "Formatted at 2026-08-08 02:57:49 on dist-test-slave-6ntk"
server_key: "a51a1933aa6955b6436e13b4798653bf"
server_key_iv: "cc8a0c47a86d89a1a436628f426812bd"
server_key_version: "encryptionkey@0"
I20260808 02:57:49.402280 10150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:49.456035 10150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:49.460996 10150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:49.461323 10150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:49.462514 10150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:49.463927 10150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:49.464039 10150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:49.464200 10150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:49.464278 10150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:49.499650 10150 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:41447
I20260808 02:57:49.499697 10309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:41447 every 8 connection(s)
I20260808 02:57:49.501085 10150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:57:49.501705 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 10150
I20260808 02:57:49.501951 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:57:49.502581 822 external_mini_cluster.cc:1428] Setting key 8f30331980437f9c6944399e53ac7995
Aug 08 02:57:49 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157869, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:57:49.522466 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:49.504159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:58431 (local address 127.0.205.190:34241)
0808 02:57:49.504710 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:49.504715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:49.506059 (+ 1344us) server_negotiation.cc:407] Connection header received
0808 02:57:49.508050 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:49.508055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:49.508142 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:49.508277 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:49.509794 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:49.510651 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:49.512247 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:49.512662 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:49.512805 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:49.516340 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:49.516345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:49.516348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:49.516381 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:49.519211 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:49.520007 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:49.520017 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:49.520020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:49.520142 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:49.520610 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:49.520616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:49.520620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:49.520894 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:49.521072 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:49.521786 (+ 714us) server_negotiation.cc:299] Negotiation successful
0808 02:57:49.522060 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":269,"threads_started":1}
I20260808 02:57:49.523488 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:49.504935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.131:58431)
0808 02:57:49.505840 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:49.505953 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:49.507744 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:49.508523 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:49.508531 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:49.509050 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:49.509608 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:49.509625 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:49.510824 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:49.510829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:49.512091 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:49.512104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:49.512927 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:49.512935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:49.513163 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:49.513818 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:49.513840 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:49.516105 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:49.519368 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:49.519377 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:49.519394 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:49.519859 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:49.520263 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:49.520268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:49.520272 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:49.520475 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:49.521033 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:49.521041 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:49.521394 (+ 353us) client_negotiation.cc:769] Sending connection context
0808 02:57:49.521682 (+ 288us) client_negotiation.cc:240] Negotiation successful
0808 02:57:49.522287 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":316,"threads_started":1}
I20260808 02:57:49.524937 10310 heartbeater.cc:347] Connected to a master server at 127.0.205.190:34241
I20260808 02:57:49.525282 10310 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:49.526136 10310 heartbeater.cc:511] Master 127.0.205.190:34241 requested a full tablet report, sending...
I20260808 02:57:49.527645 9722 ts_manager.cc:194] Registered new tserver with Master: e0e98ea59c2c4ec694d2b1c5899998d1 (127.0.205.131:41447)
I20260808 02:57:49.528510 9722 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:58431
I20260808 02:57:49.531143 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.205.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.189.keytab.
Entry for principal oryx/127.0.205.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.189.keytab.
I20260808 02:57:49.594741 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.189:35803
--webserver_interface=127.0.205.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.189.keytab
--principal=oryx/127.0.205.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:34241,127.0.205.189:35803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:57:49.954905 10140 heartbeater.cc:503] Master 127.0.205.190:34241 was elected leader, sending a full tablet report...
W20260808 02:57:49.961813 10322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:49.962194 10322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:49.962311 10322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:49.969698 10322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:49.969841 10322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:49.969967 10322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:49.970084 10322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:49.970176 10322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:49.980330 10322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:34241,127.0.205.189:35803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.189.keytab
--principal=oryx/127.0.205.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.189:35803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.205.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
W20260808 02:57:49.980748 10322 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.205.190:34241,127.0.205.189:35803'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 02:57:49.981938 10322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:49.983317 10322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:49.996129 10328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:49.996222 10329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:49.996176 10331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:49.999444 10322 server_base.cc:1034] running on GCE node
I20260808 02:57:50.000512 10322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:50.002228 10322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:50.003495 10322 hybrid_clock.cc:648] HybridClock initialized: now 1786157870003415 us; error 74 us; skew 500 ppm
I20260808 02:57:50.003877 10322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:50.007551 10322 init.cc:377] Logged in from keytab as oryx/127.0.205.189@KRBTEST.COM (short username oryx)
I20260808 02:57:50.009188 10322 webserver.cc:460] Webserver started at http://127.0.205.189:37583/ using document root <none> and password file <none>
I20260808 02:57:50.010054 10322 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:50.010193 10322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:50.010552 10322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:50.013664 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data/instance:
uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4"
format_stamp: "Formatted at 2026-08-08 02:57:50 on dist-test-slave-6ntk"
server_key: "0b971e2c3bcf3c61cc72a1cc8a19037e"
server_key_iv: "44b25a34c49417ecad7976505da14c4a"
server_key_version: "encryptionkey@0"
I20260808 02:57:50.014597 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal/instance:
uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4"
format_stamp: "Formatted at 2026-08-08 02:57:50 on dist-test-slave-6ntk"
server_key: "0b971e2c3bcf3c61cc72a1cc8a19037e"
server_key_iv: "44b25a34c49417ecad7976505da14c4a"
server_key_version: "encryptionkey@0"
I20260808 02:57:50.019979 10322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:57:50.023582 10341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:50.025064 10322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:57:50.025290 10322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal
uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4"
format_stamp: "Formatted at 2026-08-08 02:57:50 on dist-test-slave-6ntk"
server_key: "0b971e2c3bcf3c61cc72a1cc8a19037e"
server_key_iv: "44b25a34c49417ecad7976505da14c4a"
server_key_version: "encryptionkey@0"
I20260808 02:57:50.025540 10322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:50.054505 10322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:50.059322 10322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:50.059623 10322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:50.079890 10322 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.189:35803
I20260808 02:57:50.079916 10406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.189:35803 every 8 connection(s)
I20260808 02:57:50.081409 10322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
I20260808 02:57:50.085302 10407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:50.091998 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 10322
I20260808 02:57:50.092828 10407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.189@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.189@KRBTEST.COM
I20260808 02:57:50.115060 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.095941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.0.205.190:34241)
0808 02:57:50.096670 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.096679 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.097039 (+ 360us) server_negotiation.cc:407] Connection header received
0808 02:57:50.098231 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.098239 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.098318 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.098444 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.100338 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.101094 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.102853 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.103257 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.103358 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.107461 (+ 4103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.107474 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.107477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.107547 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.111253 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.112223 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.112227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.112229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.112367 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.112865 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.112871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.112878 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.113389 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.113549 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.114450 (+ 901us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.114702 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":452,"threads_started":1}
I20260808 02:57:50.116117 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.094637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.0.205.189:35803)
0808 02:57:50.095520 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.095534 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.095582 (+ 48us) server_negotiation.cc:407] Connection header received
0808 02:57:50.097932 (+ 2350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.097982 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.098275 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.098506 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.099881 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.101377 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.102731 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.103340 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.103545 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.107923 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.107934 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.107951 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.107996 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.111701 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.112558 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.112566 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.112568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.112767 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.113258 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.113263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.113266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.113927 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.114202 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.114587 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.115024 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":210,"server-negotiator.queue_time_us":671,"spinlock_wait_cycles":17664,"thread_start_us":466,"threads_started":1}
I20260808 02:57:50.116163 10414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.095883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:46816)
0808 02:57:50.096879 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.096924 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.097960 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.098968 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.098972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.099566 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.100103 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.100127 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.101237 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.101243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.102683 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.102702 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.103503 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.103511 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.103733 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.104535 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.104561 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:50.107168 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:50.111419 (+ 4251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.111430 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.111440 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.112070 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.112492 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.112498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.112502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.112724 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.113529 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:50.113536 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:50.113929 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.114307 (+ 378us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.114880 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":69,"spinlock_wait_cycles":15744,"thread_start_us":453,"threads_started":1}
I20260808 02:57:50.118221 10407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.189@KRBTEST.COM for oryx/127.0.205.189@KRBTEST.COM
I20260808 02:57:50.131134 10414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.119064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:35803 (local address 127.0.0.1:36690)
0808 02:57:50.119255 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.119274 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.119374 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.119812 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.119814 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.120160 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.120290 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.120298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.121276 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.121279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.122227 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.122238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.122869 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.122874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.123016 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.123578 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.123595 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:50.125712 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:50.128835 (+ 3123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.128839 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.128842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.129203 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.129583 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.129587 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.129589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.129741 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.130334 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:50.130337 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:50.130528 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.130645 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.130872 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 02:57:50.131381 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.119125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36690 (local address 127.0.205.189:35803)
0808 02:57:50.119362 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.119370 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.119391 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:57:50.119511 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.119515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.119584 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.119676 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.120419 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.121143 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.122358 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.122742 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.122843 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.125957 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.125963 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.125967 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.125996 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.128674 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.129362 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.129367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.129369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.129488 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.129866 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.129872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.129878 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.130227 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.130398 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.130785 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.130988 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":68}
I20260808 02:57:50.136214 10407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: Bootstrap starting.
I20260808 02:57:50.139679 10407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:50.140767 10407 log.cc:824] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:50.143635 10407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: No bootstrap required, opened a new log
I20260808 02:57:50.150884 10407 raft_consensus.cc:348] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } }
I20260808 02:57:50.151218 10407 raft_consensus.cc:374] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:50.151312 10407 raft_consensus.cc:729] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e57eb3ef7a1b4878a6c6627ddfc960e4, State: Initialized, Role: FOLLOWER
I20260808 02:57:50.151906 10407 consensus_queue.cc:260] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } }
I20260808 02:57:50.152793 10421 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } } }
I20260808 02:57:50.153093 10421 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:57:50.153860 10407 sys_catalog.cc:564] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:57:50.163532 10435 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:57:50.163720 10435 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:57:50.167205 10421 raft_consensus.cc:482] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:50.167449 10322 master_runner.cc:428] Detected that this master e57eb3ef7a1b4878a6c6627ddfc960e4 is joining an existing cluster
I20260808 02:57:50.167567 10322 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.205.189:35803
I20260808 02:57:50.167577 10421 raft_consensus.cc:504] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } }
I20260808 02:57:50.169518 10421 leader_election.cc:288] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55a2aa7994d1497196b00b86336a13a1 (127.0.205.190:34241)
I20260808 02:57:50.170168 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55a2aa7994d1497196b00b86336a13a1" is_pre_election: true
I20260808 02:57:50.171919 10343 leader_election.cc:302] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: e57eb3ef7a1b4878a6c6627ddfc960e4; no voters: 55a2aa7994d1497196b00b86336a13a1
I20260808 02:57:50.172751 10421 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
WARNING: no policy specified for oryx/127.0.205.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.188@KRBTEST.COM" created.
I20260808 02:57:50.206358 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.183694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:46826)
0808 02:57:50.184469 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.184496 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.184718 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.185326 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.185331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.186207 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.186649 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.186666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.188297 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.188304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.192929 (+ 4625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.192951 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.194466 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.194476 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.194720 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.195535 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.195565 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:50.197001 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:50.202482 (+ 5481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.202491 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.202495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.203016 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.203436 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.203441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.203445 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.203596 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.204460 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:50.204465 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:50.204703 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.204981 (+ 278us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.205337 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":106,"thread_start_us":431,"threads_started":1}
I20260808 02:57:50.211091 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.183684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46826 (local address 127.0.205.190:34241)
0808 02:57:50.183987 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.183995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.184587 (+ 592us) server_negotiation.cc:407] Connection header received
0808 02:57:50.184896 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.184905 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.185008 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.185083 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.186906 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.188094 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.193161 (+ 5067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.193985 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.194111 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.197276 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.197286 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.197294 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.197344 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.202306 (+ 4962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.203150 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.203156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.203160 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.203307 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.203777 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.203782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.203786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.204219 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.204390 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.205115 (+ 725us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.205409 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260808 02:57:50.212218 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.183654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36696 (local address 127.0.205.189:35803)
0808 02:57:50.183828 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.183834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.185016 (+ 1182us) server_negotiation.cc:407] Connection header received
0808 02:57:50.185220 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.185224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.185290 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.185355 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.187044 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.188253 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.196031 (+ 7778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.196965 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.197117 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.199718 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.199727 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.199736 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.199772 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.203572 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.204356 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.204364 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.204368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.204522 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.210018 (+ 5496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.210024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.210027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.210464 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.210642 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.211126 (+ 484us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.211745 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:57:50.213671 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.184083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:35803 (local address 127.0.0.1:36696)
0808 02:57:50.184918 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.184942 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.185100 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.185581 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.185587 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.186474 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.186866 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.186881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.188528 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.188534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.191686 (+ 3152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.191706 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.197443 (+ 5737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.197450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.197626 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.198383 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.198406 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:50.199509 (+ 1103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:50.203734 (+ 4225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.203738 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.203742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.204145 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.204661 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.204666 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.204670 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.204924 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.210627 (+ 5703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:50.210635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:50.210844 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.212715 (+ 1871us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.213103 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":473,"threads_started":1}
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157867, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.190@KRBTEST.COM for oryx/127.0.205.189@KRBTEST.COM
I20260808 02:57:50.238837 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.219309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.0.205.189:35803)
0808 02:57:50.219507 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.219513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.219620 (+ 107us) server_negotiation.cc:407] Connection header received
0808 02:57:50.220035 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.220042 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.220154 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.220284 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.222029 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.223156 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.225326 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.226179 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.226313 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.231351 (+ 5038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.231360 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.231365 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.231409 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.235480 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.236317 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.236323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.236327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.236516 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.237019 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.237025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.237028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.237394 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.237557 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.238157 (+ 600us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.238478 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 02:57:50.238850 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.218605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:35803 (local address 127.0.0.1:36698)
0808 02:57:50.219489 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.219542 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.219772 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.220370 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.220381 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.221391 (+ 1010us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.221779 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.221797 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.223319 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.223327 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.225154 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.225174 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.226386 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.226395 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.226689 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.227683 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.227715 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:50.231091 (+ 3376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:50.235672 (+ 4581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.235693 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.235700 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.236131 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.236638 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.236642 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.236644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.236888 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.237577 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:50.237586 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:50.237765 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.238047 (+ 282us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.238326 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":420,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.188.keytab.
Entry for principal oryx/127.0.205.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.188.keytab.
I20260808 02:57:50.241261 9722 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.205.189:35803: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4"
member_type: NON_VOTER
last_known_addr {
host: "127.0.205.189"
port: 35803
}
attrs {
promote: true
}
}
dest_uuid: "55a2aa7994d1497196b00b86336a13a1"
cas_config_opid_index: -1
I20260808 02:57:50.242799 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.188:41233
--webserver_interface=127.0.205.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.188.keytab
--principal=oryx/127.0.205.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:34241,127.0.205.189:35803,127.0.205.188:41233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:57:50.242985 9722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } }
I20260808 02:57:50.256403 10449 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e57eb3ef7a1b4878a6c6627ddfc960e4 (127.0.205.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } } }
I20260808 02:57:50.258211 10378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:50.261765 10449 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.205.189:35803
I20260808 02:57:50.262136 10378 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 FOLLOWER]: Refusing update from remote peer 55a2aa7994d1497196b00b86336a13a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 02:57:50.263496 10455 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:50.268520 10322 master.cc:537] Master@127.0.205.189:35803 shutting down...
W20260808 02:57:50.271319 9703 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260808 02:57:50.284757 10451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } } }
I20260808 02:57:50.285197 10451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:50.291806 10322 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:50.292436 10322 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:50.292681 10322 tablet_replica.cc:330] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: stopping tablet replica
W20260808 02:57:50.305480 9703 consensus_peers.cc:487] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 -> Peer e57eb3ef7a1b4878a6c6627ddfc960e4 (127.0.205.189:35803): Couldn't send request to peer e57eb3ef7a1b4878a6c6627ddfc960e4. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:57:50.309443 10322 master.cc:552] Master@127.0.205.189:35803 shutdown complete.
I20260808 02:57:50.309623 10322 master_runner.cc:305] Clearing existing system tablet
I20260808 02:57:50.310292 10322 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:50.310431 10322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:50.318939 10322 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.004s sys 0.005s
I20260808 02:57:50.322382 10464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:50.322814 10322 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:50.323141 10322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260808 02:57:50.323240 10322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal
uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4"
format_stamp: "Formatted at 2026-08-08 02:57:50 on dist-test-slave-6ntk"
server_key: "0b971e2c3bcf3c61cc72a1cc8a19037e"
server_key_iv: "44b25a34c49417ecad7976505da14c4a"
server_key_version: "encryptionkey@0"
I20260808 02:57:50.329450 10322 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:50.337309 10322 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 02:57:50.337538 10322 log.cc:1196] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 02:57:50.338099 10322 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: Deleting consensus metadata
I20260808 02:57:50.338445 10322 master_runner.cc:315] Copying system tablet from 127.0.205.190:34241
I20260808 02:57:50.341735 10322 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: tablet copy: Beginning tablet copy session from remote peer at address 127.0.205.190:34241
I20260808 02:57:50.367406 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.343028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46832 (local address 127.0.205.190:34241)
0808 02:57:50.343233 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.343240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.350872 (+ 7632us) server_negotiation.cc:407] Connection header received
0808 02:57:50.351130 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.351136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.351237 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.351397 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.352561 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.353910 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.356204 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.356921 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.357077 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.359561 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.359568 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.359572 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.359614 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.363868 (+ 4254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.364746 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.364753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.364766 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.364961 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.365435 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.365443 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.365448 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.365993 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.366163 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.366686 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.366943 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:57:50.367910 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.343014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:46832)
0808 02:57:50.350698 (+ 7684us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.350727 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.350918 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.351439 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.351444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.352033 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.352323 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.352336 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.354129 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.354136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.355974 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.355995 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.357099 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.357106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.357298 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.358124 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.358148 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:50.359327 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:50.364043 (+ 4716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.364050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.364055 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.364589 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.365106 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.365112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.365116 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.365294 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.366137 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:50.366143 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:50.366339 (+ 196us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.366546 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.366919 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7516,"thread_start_us":7364,"threads_started":1}
I20260808 02:57:50.369225 9747 tablet_copy_service.cc:140] P 55a2aa7994d1497196b00b86336a13a1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e57eb3ef7a1b4878a6c6627ddfc960e4 ({username='oryx', principal='oryx/127.0.205.189@KRBTEST.COM'} at 127.0.0.1:46832)
I20260808 02:57:50.369673 9747 tablet_copy_service.cc:161] P 55a2aa7994d1497196b00b86336a13a1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e57eb3ef7a1b4878a6c6627ddfc960e4 at {username='oryx', principal='oryx/127.0.205.189@KRBTEST.COM'} at 127.0.0.1:46832: session id = e57eb3ef7a1b4878a6c6627ddfc960e4-00000000000000000000000000000000
I20260808 02:57:50.374574 9747 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:57:50.377501 10322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 02:57:50.384867 10322 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: tablet copy: Starting download of 0 data blocks...
I20260808 02:57:50.385304 10322 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: tablet copy: Starting download of 1 WAL segments...
I20260808 02:57:50.389523 10322 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:57:50.394675 9747 tablet_copy_service.cc:342] P 55a2aa7994d1497196b00b86336a13a1: Request end of tablet copy session e57eb3ef7a1b4878a6c6627ddfc960e4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.205.189@KRBTEST.COM'} at 127.0.0.1:46832
I20260808 02:57:50.394973 9747 tablet_copy_service.cc:433] P 55a2aa7994d1497196b00b86336a13a1: ending tablet copy session e57eb3ef7a1b4878a6c6627ddfc960e4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e57eb3ef7a1b4878a6c6627ddfc960e4
I20260808 02:57:50.409181 10322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:50.412540 10475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:50.413823 10476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:50.421132 10322 server_base.cc:1034] running on GCE node
W20260808 02:57:50.421430 10480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:50.422055 10322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:50.422859 10322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:50.424153 10322 hybrid_clock.cc:648] HybridClock initialized: now 1786157870424093 us; error 40 us; skew 500 ppm
I20260808 02:57:50.424392 10322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:50.426895 10322 init.cc:377] Logged in from keytab as oryx/127.0.205.189@KRBTEST.COM (short username oryx)
I20260808 02:57:50.429499 10322 webserver.cc:460] Webserver started at http://127.0.205.189:44763/ using document root <none> and password file <none>
I20260808 02:57:50.430013 10322 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:50.430104 10322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:50.433297 10322 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 02:57:50.435482 10488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:50.435938 10322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 02:57:50.436110 10322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal
uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4"
format_stamp: "Formatted at 2026-08-08 02:57:50 on dist-test-slave-6ntk"
server_key: "0b971e2c3bcf3c61cc72a1cc8a19037e"
server_key_iv: "44b25a34c49417ecad7976505da14c4a"
server_key_version: "encryptionkey@0"
I20260808 02:57:50.436355 10322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:50.474704 10322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:50.479317 10322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:50.526441 10322 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.189:35803
I20260808 02:57:50.526515 10551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.189:35803 every 8 connection(s)
I20260808 02:57:50.527858 10322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
I20260808 02:57:50.532073 10310 heartbeater.cc:503] Master 127.0.205.190:34241 was elected leader, sending a full tablet report...
I20260808 02:57:50.534174 10552 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:57:50.538378 10552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: Bootstrap starting.
I20260808 02:57:50.560515 10552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:57:50.561280 10552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: Bootstrap complete.
I20260808 02:57:50.562601 10552 raft_consensus.cc:348] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } }
I20260808 02:57:50.562944 10552 raft_consensus.cc:729] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e57eb3ef7a1b4878a6c6627ddfc960e4, State: Initialized, Role: LEARNER
I20260808 02:57:50.563249 10552 consensus_queue.cc:260] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } }
I20260808 02:57:50.564101 10555 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } } }
I20260808 02:57:50.564429 10552 sys_catalog.cc:564] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:50.564597 10555 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:57:50.575613 10568 catalog_manager.cc:1259] Loaded cluster ID: 12460217a9d5465098796d959476dd8e
I20260808 02:57:50.575738 10568 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: loading cluster ID for follower catalog manager: success
I20260808 02:57:50.579543 10568 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: acquiring CA information for follower catalog manager: success
I20260808 02:57:50.582569 10568 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 02:57:50.717422 10452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:50.717743 10452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:50.717958 10452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:50.725401 10452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:50.725553 10452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:50.725648 10452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:50.725761 10452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:50.725858 10452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:50.736819 10452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:34241,127.0.205.189:35803,127.0.205.188:41233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.188.keytab
--principal=oryx/127.0.205.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.188:41233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.205.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:50.738808 10452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:50.740353 10452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:50.755076 10579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:50.755206 10578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:50.755941 10581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:50.757050 10452 server_base.cc:1034] running on GCE node
I20260808 02:57:50.757951 10452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:50.759555 10452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:50.760774 10452 hybrid_clock.cc:648] HybridClock initialized: now 1786157870760687 us; error 84 us; skew 500 ppm
I20260808 02:57:50.761134 10452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:50.764846 10452 init.cc:377] Logged in from keytab as oryx/127.0.205.188@KRBTEST.COM (short username oryx)
I20260808 02:57:50.766675 10452 webserver.cc:460] Webserver started at http://127.0.205.188:46053/ using document root <none> and password file <none>
I20260808 02:57:50.767480 10452 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:50.767632 10452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:50.767989 10452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:50.771303 10452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data/instance:
uuid: "9f21bbc67b8949eda7ab74bb20f71eba"
format_stamp: "Formatted at 2026-08-08 02:57:50 on dist-test-slave-6ntk"
server_key: "859ed8847bba94049ab960487c6bd70c"
server_key_iv: "a18a9775f6c07d44c147cdb2623aecf1"
server_key_version: "encryptionkey@0"
I20260808 02:57:50.772127 10452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal/instance:
uuid: "9f21bbc67b8949eda7ab74bb20f71eba"
format_stamp: "Formatted at 2026-08-08 02:57:50 on dist-test-slave-6ntk"
server_key: "859ed8847bba94049ab960487c6bd70c"
server_key_iv: "a18a9775f6c07d44c147cdb2623aecf1"
server_key_version: "encryptionkey@0"
I20260808 02:57:50.779402 10452 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260808 02:57:50.783566 10595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:50.785701 10452 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260808 02:57:50.785982 10452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal
uuid: "9f21bbc67b8949eda7ab74bb20f71eba"
format_stamp: "Formatted at 2026-08-08 02:57:50 on dist-test-slave-6ntk"
server_key: "859ed8847bba94049ab960487c6bd70c"
server_key_iv: "a18a9775f6c07d44c147cdb2623aecf1"
server_key_version: "encryptionkey@0"
I20260808 02:57:50.786306 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.776223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:35803 (local address 127.0.0.1:36706)
0808 02:57:50.776848 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.776874 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.777050 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.778910 (+ 1860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.778915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.778947 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:50.779176 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.779188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.781740 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.781746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.783502 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.783515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.785391 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.785397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.785577 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.785642 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.785748 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.785967 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":998,"threads_started":1}
I20260808 02:57:50.786310 10452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:50.786819 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.776755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36706 (local address 127.0.205.189:35803)
0808 02:57:50.778433 (+ 1678us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.778443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.778474 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:57:50.778566 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.778575 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.778694 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.778848 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:50.779368 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.781555 (+ 2187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.783672 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.784855 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.784995 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.785232 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.786073 (+ 841us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.786325 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1487,"thread_start_us":1285,"threads_started":1}
I20260808 02:57:50.788245 10522 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260808 02:57:50.788872 10555 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a2aa7994d1497196b00b86336a13a1. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } } }
I20260808 02:57:50.789234 10555 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:57:50.816823 10452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:50.821856 10452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:50.822256 10452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:50.844918 10452 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.188:41233
I20260808 02:57:50.844921 10660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.188:41233 every 8 connection(s)
I20260808 02:57:50.846571 10452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
I20260808 02:57:50.850636 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:50.852555 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 10452
I20260808 02:57:50.862249 10663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.188@KRBTEST.COM
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.188@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:57:50.890744 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.867319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46846 (local address 127.0.205.190:34241)
0808 02:57:50.867500 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.867506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.868524 (+ 1018us) server_negotiation.cc:407] Connection header received
0808 02:57:50.869723 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.869733 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.869849 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.870004 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.872367 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.873601 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.876669 (+ 3068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.877418 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.877570 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.883030 (+ 5460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.883039 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.883045 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.883096 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.886742 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.887690 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.887698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.887704 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.887861 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.888297 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.888304 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.888308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.888606 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.888786 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.890002 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.890292 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:57:50.892012 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.867353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:46846)
0808 02:57:50.868370 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.868422 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.869410 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.870321 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.870332 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.871413 (+ 1081us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.872053 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.872081 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.873837 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.873846 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.876477 (+ 2631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.876498 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.877704 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.877714 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.877993 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.878859 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.878902 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:50.882755 (+ 3853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:50.886884 (+ 4129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.886900 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.886912 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.887554 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.887986 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.887992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.887996 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.888159 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.888756 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:50.888765 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:50.889070 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.889421 (+ 351us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.890398 (+ 977us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":501,"thread_start_us":532,"threads_started":1}
I20260808 02:57:50.892110 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.855010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60202 (local address 127.0.205.188:41233)
0808 02:57:50.866272 (+ 11262us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.866287 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.866544 (+ 257us) server_negotiation.cc:407] Connection header received
0808 02:57:50.869410 (+ 2866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.869439 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.869751 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.869972 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.871512 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.873595 (+ 2083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.875052 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.875780 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.876011 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.881669 (+ 5658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.881678 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.881691 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.881724 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.885859 (+ 4135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.887012 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.887025 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.887032 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.887231 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.887791 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.887801 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.887808 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.888761 (+ 953us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.889226 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.889524 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.890312 (+ 788us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":10456,"thread_start_us":10240,"threads_started":1}
I20260808 02:57:50.893951 10663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.188@KRBTEST.COM for oryx/127.0.205.189@KRBTEST.COM
I20260808 02:57:50.909533 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.895051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36714 (local address 127.0.205.189:35803)
0808 02:57:50.895267 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.895273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.895490 (+ 217us) server_negotiation.cc:407] Connection header received
0808 02:57:50.895689 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.895695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.895798 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.895915 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.896953 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.897756 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.899153 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.899649 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.899799 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.903419 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.903425 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.903428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.903462 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.906577 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.907374 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.907387 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.907390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.907577 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.908000 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.908006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.908010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.908312 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.908483 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.908889 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.909132 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 02:57:50.909533 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.894993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:35803 (local address 127.0.0.1:36714)
0808 02:57:50.895375 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.895399 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.895541 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.896122 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.896127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.896627 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.896791 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.896801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.897947 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.897954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.898998 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.899010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.899811 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.899818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.900037 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.900837 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.900857 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:50.903206 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:50.906715 (+ 3509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.906722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.906727 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.907235 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.907702 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.907707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.907711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.907878 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.908445 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:50.908450 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:50.908616 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.908779 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.909129 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205}
I20260808 02:57:50.911309 10663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:57:50 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157870, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.188@KRBTEST.COM for oryx/127.0.205.188@KRBTEST.COM
I20260808 02:57:50.929414 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.912212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.0.205.188:41233)
0808 02:57:50.912495 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.912504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.912527 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:57:50.912587 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.912595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.912688 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.912801 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.913569 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.915051 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.916295 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.917174 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.917349 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.921493 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.921504 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.921513 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.921563 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.925940 (+ 4377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.926970 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.926974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.926977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.927113 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.927629 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.927632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.927634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.927913 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.928075 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.928717 (+ 642us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.929144 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":206,"server-negotiator.queue_time_us":130}
I20260808 02:57:50.929766 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.912143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.0.1:60216)
0808 02:57:50.912378 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.912395 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.912481 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.912845 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.912849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.913207 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.913408 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.913417 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.915195 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.915199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.916141 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.916153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.917361 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.917368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.917621 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.918429 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.918451 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:50.921263 (+ 2812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:50.926131 (+ 4868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.926139 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.926146 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.926841 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.927246 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.927255 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.927261 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.927503 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.928092 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:50.928101 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:50.928332 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.928549 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.928993 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":23}
I20260808 02:57:50.934739 10663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Bootstrap starting.
I20260808 02:57:50.938239 10663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:50.939358 10663 log.cc:824] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:50.942343 10663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: No bootstrap required, opened a new log
I20260808 02:57:50.950862 10663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } }
I20260808 02:57:50.951205 10663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:50.951350 10663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f21bbc67b8949eda7ab74bb20f71eba, State: Initialized, Role: FOLLOWER
I20260808 02:57:50.952020 10663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } }
I20260808 02:57:50.953125 10671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } } }
I20260808 02:57:50.953636 10671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:57:50.954140 10663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: configured and running, proceeding with master startup.
W20260808 02:57:50.966859 10684 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:57:50.967032 10684 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:57:50.969324 10452 master_runner.cc:428] Detected that this master 9f21bbc67b8949eda7ab74bb20f71eba is joining an existing cluster
I20260808 02:57:50.969470 10452 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.205.188:41233
I20260808 02:57:50.994793 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.975876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.0.205.190:34241)
0808 02:57:50.976102 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.976108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.976130 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:57:50.976195 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.976200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.976277 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.976376 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.978110 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.979309 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.981461 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.982151 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.982278 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.985414 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:50.985423 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:50.985431 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:50.985491 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:50.990071 (+ 4580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.991346 (+ 1275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.991353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.991357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.991982 (+ 625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:50.992683 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:50.992689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:50.992695 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:50.993139 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:50.993320 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:50.994135 (+ 815us) server_negotiation.cc:299] Negotiation successful
0808 02:57:50.994405 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 02:57:50.995867 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.974590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:46862)
0808 02:57:50.975444 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.975471 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.975605 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.976583 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.976588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.977524 (+ 936us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.977910 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.977925 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.979528 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.979533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.981236 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.981252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.982908 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.982920 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.983117 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.983902 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.983938 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:50.985161 (+ 1223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:50.990249 (+ 5088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.990255 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.990259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.990914 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.992159 (+ 1245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:50.992164 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:50.992167 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:50.992527 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:50.993428 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:50.993433 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:50.993684 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 02:57:50.995015 (+ 1331us) client_negotiation.cc:240] Negotiation successful
0808 02:57:50.995368 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":5064,"threads_started":1}
I20260808 02:57:51.001729 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 9800
I20260808 02:57:51.014675 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.990772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.0.1:60222)
0808 02:57:50.996165 (+ 5393us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.996209 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.996361 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.996797 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.996802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.997337 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.997546 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.997559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.999092 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.999097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.000685 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.000710 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.002054 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.002065 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.002239 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.004211 (+ 1972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:51.004233 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:51.005423 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:51.011139 (+ 5716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.011144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.011148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.011794 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.012224 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.012229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.012233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.012416 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.013540 (+ 1124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:51.013545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:51.013687 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 02:57:51.013932 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 02:57:51.014226 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5244,"mutex_wait_us":1,"thread_start_us":21021,"threads_started":1}
I20260808 02:57:51.014806 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.976303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36728 (local address 127.0.205.189:35803)
0808 02:57:50.976541 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.976550 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.990241 (+ 13691us) server_negotiation.cc:407] Connection header received
0808 02:57:50.991236 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.991246 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.991356 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.991477 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.993581 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.995035 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.997355 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.998030 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.998132 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.001344 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:51.001351 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:51.001355 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:51.001395 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:51.005478 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.011558 (+ 6080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.011568 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.011574 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.011802 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.012354 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.012362 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.012368 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.012899 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:51.013105 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:51.013956 (+ 851us) server_negotiation.cc:299] Negotiation successful
0808 02:57:51.014287 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:57:51.015738 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.990988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60222 (local address 127.0.205.188:41233)
0808 02:57:50.991173 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:50.991179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:50.996273 (+ 5094us) server_negotiation.cc:407] Connection header received
0808 02:57:50.996490 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:50.996495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:50.996564 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:50.996625 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:50.997705 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.998890 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.000870 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.001644 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.001746 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.005608 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:51.005615 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:51.005620 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:51.005655 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:51.010965 (+ 5310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.011939 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.011945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.011948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.012100 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.012537 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.012542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.012546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.013331 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:51.013478 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:51.015011 (+ 1533us) server_negotiation.cc:299] Negotiation successful
0808 02:57:51.015286 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":53}
I20260808 02:57:51.016283 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:50.975451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:35803 (local address 127.0.0.1:36728)
0808 02:57:50.990131 (+ 14680us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:50.990154 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:50.991064 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:50.991737 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:50.991746 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:50.992824 (+ 1078us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:50.993336 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.993352 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.995241 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.995245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.997170 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:50.997184 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:50.998428 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:50.998440 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:50.999114 (+ 674us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:50.999920 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:50.999949 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:51.001118 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:51.005632 (+ 4514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.005636 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.005640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.011394 (+ 5754us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.011950 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.011956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.011963 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.012197 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.013225 (+ 1028us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:51.013232 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:51.013524 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 02:57:51.015027 (+ 1503us) client_negotiation.cc:240] Negotiation successful
0808 02:57:51.015569 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14489,"mutex_wait_us":156,"thread_start_us":14810,"threads_started":1}
Aug 08 02:57:51 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.129@KRBTEST.COM
Aug 08 02:57:51 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157867, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.190@KRBTEST.COM for oryx/127.0.205.188@KRBTEST.COM
I20260808 02:57:51.035615 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.003977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49854 (local address 127.0.205.129:46555)
0808 02:57:51.017562 (+ 13585us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:51.017572 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:51.017635 (+ 63us) server_negotiation.cc:407] Connection header received
0808 02:57:51.017721 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:51.017732 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:51.018061 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:51.018348 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:51.019755 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.020824 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.022147 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.022725 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.022872 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.028405 (+ 5533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:51.028422 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:51.028443 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:51.028497 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:51.032226 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.033002 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.033011 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.033013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.033150 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.033673 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.033678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.033682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.034190 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:51.034529 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:51.034879 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 02:57:51.035190 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13389,"thread_start_us":14567,"threads_started":1}
I20260808 02:57:51.037283 9960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:51.042891 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.005578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.0.1:60232)
0808 02:57:51.005971 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:51.005993 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:51.006131 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:51.007936 (+ 1805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:51.007942 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:51.008429 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:51.008960 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.008976 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.028480 (+ 19504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.028487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.030043 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.030066 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.031146 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.031153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.031346 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.031988 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:51.032015 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:51.034781 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:51.039678 (+ 4897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.039687 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.039695 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.040440 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.041034 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.041039 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.041043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.041200 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.041770 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:51.041775 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:51.041977 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 02:57:51.042128 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 02:57:51.042449 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260808 02:57:51.042909 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.005934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60232 (local address 127.0.205.188:41233)
0808 02:57:51.006812 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:51.006817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:51.006841 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:57:51.006949 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:51.006954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:51.007037 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:51.009133 (+ 2096us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:51.027418 (+ 18285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.028299 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.030228 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.030959 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.031091 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.035055 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:51.035062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:51.035068 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:51.035115 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:51.039384 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.040693 (+ 1309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.040701 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.040708 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.040900 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.041329 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.041334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.041338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.041644 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:51.041811 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:51.042238 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 02:57:51.042448 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":1271,"threads_started":1}
I20260808 02:57:51.044309 9722 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.205.188:41233: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba"
member_type: NON_VOTER
last_known_addr {
host: "127.0.205.188"
port: 41233
}
attrs {
promote: true
}
}
dest_uuid: "55a2aa7994d1497196b00b86336a13a1"
cas_config_opid_index: 5
I20260808 02:57:51.045125 9722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } }
I20260808 02:57:51.047149 10451 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9f21bbc67b8949eda7ab74bb20f71eba (127.0.205.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.047938 10634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:51.051162 10451 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.205.188:41233
I20260808 02:57:51.051784 10634 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Refusing update from remote peer 55a2aa7994d1497196b00b86336a13a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 02:57:51.051864 10452 master.cc:537] Master@127.0.205.188:41233 shutting down...
I20260808 02:57:51.053357 10522 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 LEARNER]: Refusing update from remote peer 55a2aa7994d1497196b00b86336a13a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 02:57:51.057120 10671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:51.057468 10671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } }
I20260808 02:57:51.059471 10671 leader_election.cc:288] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 55a2aa7994d1497196b00b86336a13a1 (127.0.205.190:34241), e57eb3ef7a1b4878a6c6627ddfc960e4 (127.0.205.189:35803)
I20260808 02:57:51.060223 9736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55a2aa7994d1497196b00b86336a13a1" is_pre_election: true
I20260808 02:57:51.061749 10522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" is_pre_election: true
I20260808 02:57:51.066138 10599 leader_election.cc:302] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f21bbc67b8949eda7ab74bb20f71eba; no voters: 55a2aa7994d1497196b00b86336a13a1, e57eb3ef7a1b4878a6c6627ddfc960e4
I20260808 02:57:51.066275 10702 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:51.066869 10671 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:57:51.072921 10451 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 02:57:51.074635 10452 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:51.075080 10452 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:51.074884 10704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.075268 10704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:51.075346 10452 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: stopping tablet replica
I20260808 02:57:51.085574 10452 master.cc:552] Master@127.0.205.188:41233 shutdown complete.
I20260808 02:57:51.085685 10452 master_runner.cc:305] Clearing existing system tablet
I20260808 02:57:51.086222 10452 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:51.086351 10452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:51.091653 10452 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 02:57:51.099249 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.097825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.0.1:60240)
0808 02:57:51.098776 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:51.098968 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:41233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":762}
W20260808 02:57:51.099764 9703 consensus_peers.cc:487] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 -> Peer 9f21bbc67b8949eda7ab74bb20f71eba (127.0.205.188:41233): Couldn't send request to peer 9f21bbc67b8949eda7ab74bb20f71eba. Status: Network error: Client connection negotiation failed: client connection to 127.0.205.188:41233: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:57:51.107189 10522 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9f21bbc67b8949eda7ab74bb20f71eba (127.0.205.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.114126 10710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:51.114686 10452 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:51.114992 10452 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.011s sys 0.011s
I20260808 02:57:51.114765 10555 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: NON_VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: true } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.115108 10555 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:57:51.115110 10452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal
uuid: "9f21bbc67b8949eda7ab74bb20f71eba"
format_stamp: "Formatted at 2026-08-08 02:57:50 on dist-test-slave-6ntk"
server_key: "859ed8847bba94049ab960487c6bd70c"
server_key_iv: "a18a9775f6c07d44c147cdb2623aecf1"
server_key_version: "encryptionkey@0"
I20260808 02:57:51.116376 10704 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1: attempting to promote NON_VOTER e57eb3ef7a1b4878a6c6627ddfc960e4 to VOTER
I20260808 02:57:51.117368 10704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } }
I20260808 02:57:51.119742 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.119255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.0.1:60248)
0808 02:57:51.119486 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:51.119590 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:41233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 02:57:51.120518 10521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 LEARNER]: Refusing update from remote peer 55a2aa7994d1497196b00b86336a13a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 02:57:51.120939 10452 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:51.121110 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.120661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.0.1:60254)
0808 02:57:51.120852 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:51.120940 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:41233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260808 02:57:51.121302 9703 consensus_peers.cc:487] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 -> Peer 9f21bbc67b8949eda7ab74bb20f71eba (127.0.205.188:41233): Couldn't send request to peer 9f21bbc67b8949eda7ab74bb20f71eba. Status: Network error: Client connection negotiation failed: client connection to 127.0.205.188:41233: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:57:51.121649 10704 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 02:57:51.124208 10703 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e57eb3ef7a1b4878a6c6627ddfc960e4 (127.0.205.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.128507 10452 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 02:57:51.128430 10702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.128687 10452 log.cc:1196] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 02:57:51.128784 10702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:51.129436 10452 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Deleting consensus metadata
I20260808 02:57:51.129752 10452 master_runner.cc:315] Copying system tablet from 127.0.205.190:34241
I20260808 02:57:51.129976 10521 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e57eb3ef7a1b4878a6c6627ddfc960e4 (127.0.205.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.135172 10702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.135779 10702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:51.136746 10555 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.137136 10555 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:57:51.145835 10452 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: tablet copy: Beginning tablet copy session from remote peer at address 127.0.205.190:34241
I20260808 02:57:51.166031 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.146935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46874 (local address 127.0.205.190:34241)
0808 02:57:51.147224 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:51.147231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:51.148687 (+ 1456us) server_negotiation.cc:407] Connection header received
0808 02:57:51.148951 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:51.148957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:51.149033 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:51.149146 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:51.150854 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.151864 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.153634 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.154487 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.154643 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.157996 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:51.158003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:51.158008 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:51.158053 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:51.162408 (+ 4355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.163161 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.163170 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.163175 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.163414 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.163904 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.163910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.163915 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.164287 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:51.164457 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:51.164974 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 02:57:51.165292 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260808 02:57:51.166584 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.147042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:46874)
0808 02:57:51.148539 (+ 1497us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:51.148578 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:51.148804 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:51.149176 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:51.149181 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:51.150055 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:51.150688 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.150709 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.152055 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.152059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.153460 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.153473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.154859 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.154888 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.155166 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.156103 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:51.156122 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:51.157716 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:51.162556 (+ 4840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.162562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.162567 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.163033 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.163540 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.163545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.163549 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.163787 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.164418 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:51.164423 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:51.164714 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 02:57:51.164878 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 02:57:51.165305 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1358,"thread_start_us":750,"threads_started":1}
I20260808 02:57:51.167398 9747 tablet_copy_service.cc:140] P 55a2aa7994d1497196b00b86336a13a1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9f21bbc67b8949eda7ab74bb20f71eba ({username='oryx', principal='oryx/127.0.205.188@KRBTEST.COM'} at 127.0.0.1:46874)
I20260808 02:57:51.167671 9747 tablet_copy_service.cc:161] P 55a2aa7994d1497196b00b86336a13a1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9f21bbc67b8949eda7ab74bb20f71eba at {username='oryx', principal='oryx/127.0.205.188@KRBTEST.COM'} at 127.0.0.1:46874: session id = 9f21bbc67b8949eda7ab74bb20f71eba-00000000000000000000000000000000
I20260808 02:57:51.171469 9747 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:57:51.173736 10452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 02:57:51.181044 10452 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: tablet copy: Starting download of 0 data blocks...
I20260808 02:57:51.181466 10452 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: tablet copy: Starting download of 1 WAL segments...
I20260808 02:57:51.185226 10452 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:57:51.186429 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 9800
I20260808 02:57:51.195626 9747 tablet_copy_service.cc:342] P 55a2aa7994d1497196b00b86336a13a1: Request end of tablet copy session 9f21bbc67b8949eda7ab74bb20f71eba-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.205.188@KRBTEST.COM'} at 127.0.0.1:46874
I20260808 02:57:51.195914 9747 tablet_copy_service.cc:433] P 55a2aa7994d1497196b00b86336a13a1: ending tablet copy session 9f21bbc67b8949eda7ab74bb20f71eba-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9f21bbc67b8949eda7ab74bb20f71eba
I20260808 02:57:51.208575 10452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:51.211447 10722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:51.215870 10723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:51.216987 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:46555
--local_ip_for_outbound_sockets=127.0.205.129
--tserver_master_addrs=127.0.205.190:34241
--webserver_port=45709
--webserver_interface=127.0.205.129
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab
--principal=oryx/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.205.190:34241,127.0.205.189:35803
--tserver_master_addrs=127.0.205.190:34241,127.0.205.189:35803,127.0.205.188:41233 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:51.217514 10725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:51.223590 10452 server_base.cc:1034] running on GCE node
I20260808 02:57:51.223994 10452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:51.224727 10452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:51.225962 10452 hybrid_clock.cc:648] HybridClock initialized: now 1786157871225902 us; error 49 us; skew 500 ppm
I20260808 02:57:51.226243 10452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:51 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157871, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:51.228583 10452 init.cc:377] Logged in from keytab as oryx/127.0.205.188@KRBTEST.COM (short username oryx)
I20260808 02:57:51.234148 10452 webserver.cc:460] Webserver started at http://127.0.205.188:45693/ using document root <none> and password file <none>
I20260808 02:57:51.234551 10452 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:51.234660 10452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:51.242642 10452 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.007s
I20260808 02:57:51.245014 10738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:51.245484 10452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 02:57:51.245654 10452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal
uuid: "9f21bbc67b8949eda7ab74bb20f71eba"
format_stamp: "Formatted at 2026-08-08 02:57:50 on dist-test-slave-6ntk"
server_key: "859ed8847bba94049ab960487c6bd70c"
server_key_iv: "a18a9775f6c07d44c147cdb2623aecf1"
server_key_version: "encryptionkey@0"
I20260808 02:57:51.245939 10452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:51.256896 10452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:51.261096 10452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:51.302232 10452 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.188:41233
I20260808 02:57:51.302309 10802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.188:41233 every 8 connection(s)
I20260808 02:57:51.303493 10452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
I20260808 02:57:51.309417 10803 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:57:51.313372 10803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Bootstrap starting.
I20260808 02:57:51.333420 10803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:57:51.334012 10803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Bootstrap complete.
I20260808 02:57:51.335184 10803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } }
I20260808 02:57:51.335503 10803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9f21bbc67b8949eda7ab74bb20f71eba, State: Initialized, Role: LEARNER
I20260808 02:57:51.335786 10803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } }
I20260808 02:57:51.336855 10809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.337196 10803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:51.338332 10809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: This master's current role is: LEARNER
I20260808 02:57:51.367695 10821 catalog_manager.cc:1259] Loaded cluster ID: 12460217a9d5465098796d959476dd8e
I20260808 02:57:51.367945 10821 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: loading cluster ID for follower catalog manager: success
I20260808 02:57:51.371620 10821 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: acquiring CA information for follower catalog manager: success
I20260808 02:57:51.374178 10821 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 02:57:51.675978 10730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:51.676290 10730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:51.676388 10730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:51.683910 10730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:51.684088 10730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:51.684329 10730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:51.694519 10730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab
--principal=oryx/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:46555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=45709
--tserver_master_addrs=127.0.205.190:34241,127.0.205.189:35803,127.0.205.188:41233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:51.695994 10730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:51.697358 10730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:51.711633 10829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:51.711683 10833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:51.711719 10830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:51.712579 10730 server_base.cc:1034] running on GCE node
I20260808 02:57:51.713292 10730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:51.714375 10730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:51.715626 10730 hybrid_clock.cc:648] HybridClock initialized: now 1786157871715572 us; error 42 us; skew 500 ppm
I20260808 02:57:51.716063 10730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:51 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157871, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:51.719662 10730 init.cc:377] Logged in from keytab as oryx/127.0.205.129@KRBTEST.COM (short username oryx)
I20260808 02:57:51.721263 10730 webserver.cc:460] Webserver started at http://127.0.205.129:45709/ using document root <none> and password file <none>
I20260808 02:57:51.722162 10730 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:51.722334 10730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:51.727975 10730 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260808 02:57:51.731055 10841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:51.732411 10730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:57:51.732595 10730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "17cab275b91140e5bbcea6f6bd266e07"
format_stamp: "Formatted at 2026-08-08 02:57:48 on dist-test-slave-6ntk"
server_key: "cd7bebb928a722e27381690d005cbbc2"
server_key_iv: "97f65b8fa41e789553ef2e4327a5099f"
server_key_version: "encryptionkey@0"
I20260808 02:57:51.733618 10730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:51.745645 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.738389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.0.1:58410)
0808 02:57:51.738923 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:51.738950 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:51.739108 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:51.739772 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:51.739777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:51.739806 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:51.740086 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.740103 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.742039 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.742046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.743341 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.743355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.744965 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.744970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.745121 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.745147 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:57:51.745286 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 02:57:51.745395 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":270,"threads_started":1}
I20260808 02:57:51.745774 10843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.738565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58410 (local address 127.0.205.188:41233)
0808 02:57:51.739306 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:51.739311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:51.739335 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:57:51.739441 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:51.739449 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:51.739563 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:51.739713 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:51.740286 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.741808 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.743482 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.744786 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.744918 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.745125 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:51.745331 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 02:57:51.745479 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":443,"threads_started":1}
I20260808 02:57:51.746840 10770 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 02:57:51.747499 10809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a2aa7994d1497196b00b86336a13a1. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:51.747855 10809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: This master's current role is: LEARNER
I20260808 02:57:51.753998 10730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:51.758843 10730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:51.759174 10730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:51.760275 10730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:51.761704 10730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:51.761843 10730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:51.762049 10730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:51.762151 10730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:51.798991 10730 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:46555
I20260808 02:57:51.799052 10993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:46555 every 8 connection(s)
I20260808 02:57:51.800603 10730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:51.808717 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 10730
I20260808 02:57:51.809099 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 9978
Aug 08 02:57:51 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157871, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.189@KRBTEST.COM
Aug 08 02:57:51 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157871, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
Aug 08 02:57:51 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157871, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.188@KRBTEST.COM
Aug 08 02:57:51 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.130@KRBTEST.COM
I20260808 02:57:51.830907 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.803682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:36061 (local address 127.0.205.189:35803)
0808 02:57:51.804288 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:51.804295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:51.805536 (+ 1241us) server_negotiation.cc:407] Connection header received
0808 02:57:51.808324 (+ 2788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:51.808330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:51.808431 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:51.808519 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:51.812187 (+ 3668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.813389 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.816108 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.816779 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.816881 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.822397 (+ 5516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:51.822406 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:51.822411 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:51.822477 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:51.826504 (+ 4027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.827654 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.827662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.827668 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.827822 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.828442 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.828448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.828453 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.828884 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:51.829043 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:51.830235 (+ 1192us) server_negotiation.cc:299] Negotiation successful
0808 02:57:51.830497 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":289,"threads_started":1}
I20260808 02:57:51.831020 10843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.806830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:47467 (local address 127.0.205.188:41233)
0808 02:57:51.807051 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:51.807059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:51.807543 (+ 484us) server_negotiation.cc:407] Connection header received
0808 02:57:51.808347 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:51.808356 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:51.808451 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:51.808535 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:51.812182 (+ 3647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.813338 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.816108 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.816774 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.816887 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.823537 (+ 6650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:51.823545 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:51.823551 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:51.823596 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:51.827491 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.828613 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.828618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.828622 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.828767 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.829298 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.829304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.829307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.829720 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:51.829844 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:51.830442 (+ 598us) server_negotiation.cc:299] Negotiation successful
0808 02:57:51.830674 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:57:51.831072 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.804243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:36633 (local address 127.0.205.190:34241)
0808 02:57:51.804960 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:51.804968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:51.806658 (+ 1690us) server_negotiation.cc:407] Connection header received
0808 02:57:51.808324 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:51.808330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:51.808431 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:51.808519 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:51.811651 (+ 3132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.813000 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.816113 (+ 3113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.816779 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.816888 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.823362 (+ 6474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:51.823370 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:51.823380 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:51.823451 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:51.827486 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.828578 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.828583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.828586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.828752 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.829298 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.829304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.829307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.829673 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:51.829817 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:51.830442 (+ 625us) server_negotiation.cc:299] Negotiation successful
0808 02:57:51.830674 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":287,"threads_started":1}
I20260808 02:57:51.833566 11003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.804388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:35803 (local address 127.0.205.129:36061)
0808 02:57:51.805369 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:51.805409 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:51.807979 (+ 2570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:51.809107 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:51.809119 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:51.811690 (+ 2571us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:51.811993 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.812006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.813533 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.813539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.815936 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.815951 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.817266 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.817274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.817697 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.818655 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:51.818691 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:51.822090 (+ 3399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:51.826787 (+ 4697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.826797 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.826817 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.827494 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.827996 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.828000 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.828005 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.828278 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.829155 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:51.829164 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:51.829678 (+ 514us) client_negotiation.cc:769] Sending connection context
0808 02:57:51.831678 (+ 2000us) client_negotiation.cc:240] Negotiation successful
0808 02:57:51.832244 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":567,"thread_start_us":1632,"threads_started":1}
I20260808 02:57:51.833671 11005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.806726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.205.129:47467)
0808 02:57:51.807444 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:51.807458 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:51.807986 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:51.809108 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:51.809120 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:51.811050 (+ 1930us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:51.811482 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.811495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.813533 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.813539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.815936 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.815951 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.817267 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.817274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.817719 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.818650 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:51.818703 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:51.823313 (+ 4610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:51.827676 (+ 4363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.827686 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.827690 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.828434 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.828918 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.828923 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.828927 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.829163 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.829939 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:51.829943 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:51.830119 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 02:57:51.831375 (+ 1256us) client_negotiation.cc:240] Negotiation successful
0808 02:57:51.832048 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":525,"spinlock_wait_cycles":92288,"thread_start_us":381,"threads_started":1}
I20260808 02:57:51.833684 11002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.804386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.129:36633)
0808 02:57:51.806542 (+ 2156us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:51.806562 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:51.807979 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:51.809106 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:51.809119 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:51.810188 (+ 1069us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:51.811449 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.811474 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.813221 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.813227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.815933 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:51.815948 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.817310 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.817317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:51.817683 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.818852 (+ 1169us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:51.818878 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:51.823098 (+ 4220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:51.827689 (+ 4591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.827693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.827696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.828430 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.828918 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:51.828923 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:51.828927 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:51.829163 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:51.829937 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:51.829942 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:51.830119 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 02:57:51.831281 (+ 1162us) client_negotiation.cc:240] Negotiation successful
0808 02:57:51.831931 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1967,"mutex_wait_us":382,"spinlock_wait_cycles":31616,"thread_start_us":492,"threads_started":1}
I20260808 02:57:51.835703 10996 heartbeater.cc:347] Connected to a master server at 127.0.205.190:34241
I20260808 02:57:51.835850 10995 heartbeater.cc:347] Connected to a master server at 127.0.205.188:41233
I20260808 02:57:51.835884 10997 heartbeater.cc:347] Connected to a master server at 127.0.205.189:35803
I20260808 02:57:51.836093 10995 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:51.836099 10997 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:51.836093 10996 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:51.837500 10995 heartbeater.cc:511] Master 127.0.205.188:41233 requested a full tablet report, sending...
I20260808 02:57:51.837553 10996 heartbeater.cc:511] Master 127.0.205.190:34241 requested a full tablet report, sending...
I20260808 02:57:51.837496 10997 heartbeater.cc:511] Master 127.0.205.189:35803 requested a full tablet report, sending...
I20260808 02:57:51.837597 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:51.810271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.0.205.130:33955)
0808 02:57:51.811340 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:51.811349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:51.811394 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:57:51.811478 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:51.811485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:51.811744 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:51.811917 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:51.814051 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.815941 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.817900 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:51.819405 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:51.826901 (+ 7496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:51.827386 (+ 485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:51.827416 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:51.827435 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:51.827477 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:51.831662 (+ 4185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.832650 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.832659 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.832662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.832820 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:51.833304 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:51.833309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:51.833311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:51.833786 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:51.834137 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:51.836801 (+ 2664us) server_negotiation.cc:299] Negotiation successful
0808 02:57:51.837149 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":712,"threads_started":1}
I20260808 02:57:51.839491 10133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:51.840350 9722 ts_manager.cc:194] Re-registered known tserver with Master: 17cab275b91140e5bbcea6f6bd266e07 (127.0.205.129:46555)
I20260808 02:57:51.840679 10510 ts_manager.cc:194] Registered new tserver with Master: 17cab275b91140e5bbcea6f6bd266e07 (127.0.205.129:46555)
I20260808 02:57:51.840687 10760 ts_manager.cc:194] Registered new tserver with Master: 17cab275b91140e5bbcea6f6bd266e07 (127.0.205.129:46555)
I20260808 02:57:51.841542 9722 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:36633
I20260808 02:57:51.926811 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 9978
I20260808 02:57:51.951844 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:33955
--local_ip_for_outbound_sockets=127.0.205.130
--tserver_master_addrs=127.0.205.190:34241
--webserver_port=43125
--webserver_interface=127.0.205.130
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab
--principal=oryx/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.205.190:34241,127.0.205.189:35803
--tserver_master_addrs=127.0.205.190:34241,127.0.205.189:35803,127.0.205.188:41233 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:57:52.233435 10704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: NON_VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: true } } }
I20260808 02:57:52.234292 10704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:52.235507 10704 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1: attempting to promote NON_VOTER 9f21bbc67b8949eda7ab74bb20f71eba to VOTER
I20260808 02:57:52.236402 10704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } }
I20260808 02:57:52.239782 10770 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 LEARNER]: Refusing update from remote peer 55a2aa7994d1497196b00b86336a13a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 02:57:52.240602 10704 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 02:57:52.240890 10521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 FOLLOWER]: Refusing update from remote peer 55a2aa7994d1497196b00b86336a13a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 02:57:52.242033 11015 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 02:57:52.245213 10704 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9f21bbc67b8949eda7ab74bb20f71eba (127.0.205.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:52.245982 10521 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9f21bbc67b8949eda7ab74bb20f71eba (127.0.205.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:52.248617 10704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:52.248574 11014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:52.248870 10704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:52.248903 11014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:52.249800 10704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:52.250262 10704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:52.251543 10770 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9f21bbc67b8949eda7ab74bb20f71eba (127.0.205.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:52.255504 11020 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:52.255999 11020 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:57:52.258378 11021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55a2aa7994d1497196b00b86336a13a1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:52.258797 11021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:57:52.315157 11012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:52.315464 11012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:52.315653 11012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:52.323170 11012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:52.323329 11012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:52.323511 11012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:52.334255 11012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab
--principal=oryx/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:33955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=43125
--tserver_master_addrs=127.0.205.190:34241,127.0.205.189:35803,127.0.205.188:41233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:52.335878 11012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:52.337499 11012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:52.353271 11028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:52.353368 11030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:52.353322 11027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:52.354393 11012 server_base.cc:1034] running on GCE node
I20260808 02:57:52.355152 11012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:52.356256 11012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:52.357519 11012 hybrid_clock.cc:648] HybridClock initialized: now 1786157872357469 us; error 69 us; skew 500 ppm
I20260808 02:57:52.357928 11012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:52 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:52.361694 11012 init.cc:377] Logged in from keytab as oryx/127.0.205.130@KRBTEST.COM (short username oryx)
I20260808 02:57:52.363435 11012 webserver.cc:460] Webserver started at http://127.0.205.130:43125/ using document root <none> and password file <none>
I20260808 02:57:52.364275 11012 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:52.364470 11012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:52.370818 11012 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:57:52.373714 11037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:52.375509 11012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:57:52.375715 11012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "be391462969e4dad920ea4aa06ee1169"
format_stamp: "Formatted at 2026-08-08 02:57:48 on dist-test-slave-6ntk"
server_key: "d93506c411e4a43447e33c6864714c0b"
server_key_iv: "67688e942e6faa612e285ba8921321ee"
server_key_version: "encryptionkey@0"
I20260808 02:57:52.376801 11012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:52.400039 11012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:52.404366 11012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:52.404609 11012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:52.405618 11012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:52.407384 11012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:52.407516 11012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:52.407644 11012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:52.407728 11012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:52.442858 11012 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:33955
I20260808 02:57:52.442961 11188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:33955 every 8 connection(s)
I20260808 02:57:52.444379 11012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:52.449060 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 11012
I20260808 02:57:52.449445 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 10150
Aug 08 02:57:52 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.189@KRBTEST.COM
Aug 08 02:57:52 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
Aug 08 02:57:52 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.131@KRBTEST.COM
I20260808 02:57:52.485121 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:52.447462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:40949 (local address 127.0.205.189:35803)
0808 02:57:52.463032 (+ 15570us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:52.463039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:52.463064 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:57:52.463283 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:52.463288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:52.463406 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:52.463529 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:52.466099 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.467385 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.470058 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.470713 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.470821 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:52.476529 (+ 5708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:52.476538 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:52.476544 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:52.476597 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:52.480701 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:52.481747 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:52.481753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:52.481756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:52.481936 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:52.482532 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:52.482538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:52.482541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:52.483018 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:52.483188 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:52.484536 (+ 1348us) server_negotiation.cc:299] Negotiation successful
0808 02:57:52.484788 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15424,"thread_start_us":15595,"threads_started":1}
I20260808 02:57:52.486786 11196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:52.447523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:41155 (local address 127.0.205.190:34241)
0808 02:57:52.463742 (+ 16219us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:52.463748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:52.466152 (+ 2404us) server_negotiation.cc:407] Connection header received
0808 02:57:52.466445 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:52.466451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:52.466550 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:52.466629 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:52.468186 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.469291 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.471279 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.471970 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.472042 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:52.477024 (+ 4982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:52.477031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:52.477036 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:52.477085 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:52.481748 (+ 4663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:52.482744 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:52.482750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:52.482753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:52.482946 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:52.483568 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:52.483575 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:52.483579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:52.483990 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:52.484166 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:52.486102 (+ 1936us) server_negotiation.cc:299] Negotiation successful
0808 02:57:52.486376 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16089,"thread_start_us":15951,"threads_started":1}
Aug 08 02:57:52 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.188@KRBTEST.COM
I20260808 02:57:52.487988 11197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:52.448226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:35803 (local address 127.0.205.130:40949)
0808 02:57:52.458405 (+ 10179us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:52.459602 (+ 1197us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:52.462958 (+ 3356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:52.463980 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:52.463989 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:52.464825 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:52.465701 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:52.465747 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.467610 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.467615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:52.469890 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:52.469905 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.471161 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.471171 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:52.471450 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:52.472424 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:52.472454 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:52.476237 (+ 3783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:52.480899 (+ 4662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:52.480908 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:52.480924 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:52.481569 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:52.482080 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:52.482086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:52.482092 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:52.482388 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:52.483280 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:52.483288 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:52.483766 (+ 478us) client_negotiation.cc:769] Sending connection context
0808 02:57:52.484108 (+ 342us) client_negotiation.cc:240] Negotiation successful
0808 02:57:52.487177 (+ 3069us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9718,"mutex_wait_us":2478,"spinlock_wait_cycles":67328,"thread_start_us":17273,"threads_started":1}
I20260808 02:57:52.488049 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:52.448226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.130:41155)
0808 02:57:52.466048 (+ 17822us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:52.466073 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:52.466268 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:52.466854 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:52.466860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:52.467597 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:52.467995 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:52.468007 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.469502 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.469507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:52.471112 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:52.471123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.472445 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.472452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:52.472663 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:52.473482 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:52.473503 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:52.476799 (+ 3296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:52.481951 (+ 5152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:52.481957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:52.481961 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:52.482580 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:52.483098 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:52.483103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:52.483107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:52.483398 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:52.484282 (+ 884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:52.484288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:52.484532 (+ 244us) client_negotiation.cc:769] Sending connection context
0808 02:57:52.485585 (+ 1053us) client_negotiation.cc:240] Negotiation successful
0808 02:57:52.486100 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17467,"mutex_wait_us":48,"thread_start_us":10132,"threads_started":1}
I20260808 02:57:52.489813 11191 heartbeater.cc:347] Connected to a master server at 127.0.205.190:34241
I20260808 02:57:52.490178 11191 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:52.489840 11192 heartbeater.cc:347] Connected to a master server at 127.0.205.189:35803
I20260808 02:57:52.490826 11192 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:52.491138 11191 heartbeater.cc:511] Master 127.0.205.190:34241 requested a full tablet report, sending...
I20260808 02:57:52.491461 11192 heartbeater.cc:511] Master 127.0.205.189:35803 requested a full tablet report, sending...
I20260808 02:57:52.493075 9722 ts_manager.cc:194] Re-registered known tserver with Master: be391462969e4dad920ea4aa06ee1169 (127.0.205.130:33955)
I20260808 02:57:52.493326 10510 ts_manager.cc:194] Registered new tserver with Master: be391462969e4dad920ea4aa06ee1169 (127.0.205.130:33955)
I20260808 02:57:52.494513 9722 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:41155
I20260808 02:57:52.495343 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:52.450624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42570 (local address 127.0.205.131:41447)
0808 02:57:52.464837 (+ 14213us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:52.464846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:52.465471 (+ 625us) server_negotiation.cc:407] Connection header received
0808 02:57:52.465726 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:52.465738 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:52.466157 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:52.466429 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:52.469908 (+ 3479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.472663 (+ 2755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.475747 (+ 3084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.476631 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.476754 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:52.483888 (+ 7134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:52.483902 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:52.483920 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:52.483964 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:52.488627 (+ 4663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:52.489545 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:52.489553 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:52.489556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:52.492186 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:52.492738 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:52.492743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:52.492746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:52.493186 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:52.493514 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:52.494788 (+ 1274us) server_negotiation.cc:299] Negotiation successful
0808 02:57:52.495002 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14024,"thread_start_us":13838,"threads_started":1}
I20260808 02:57:52.496948 10304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:52.497428 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:52.466572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:47715 (local address 127.0.205.188:41233)
0808 02:57:52.473322 (+ 6750us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:52.473328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:52.476435 (+ 3107us) server_negotiation.cc:407] Connection header received
0808 02:57:52.476916 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:52.476925 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:52.477020 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:52.477107 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:52.478913 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.480425 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.482936 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.483669 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.483789 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:52.489742 (+ 5953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:52.489754 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:52.489760 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:52.489807 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:52.494293 (+ 4486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:52.495356 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:52.495364 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:52.495370 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:52.495532 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:52.495983 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:52.495989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:52.495994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:52.496270 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:52.496435 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:52.496829 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 02:57:52.497051 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6606,"thread_start_us":6464,"threads_started":1}
I20260808 02:57:52.497534 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:52.472629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.205.130:47715)
0808 02:57:52.476307 (+ 3678us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:52.476330 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:52.476729 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:52.477395 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:52.477400 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:52.478173 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:52.478643 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:52.478657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.480638 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.480646 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:52.482760 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:52.482774 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:52.484151 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:52.484158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:52.484473 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:52.485274 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:52.485296 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:52.489474 (+ 4178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:52.494526 (+ 5052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:52.494536 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:52.494542 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:52.495175 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:52.495675 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:52.495681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:52.495684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:52.495852 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:52.496398 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:52.496402 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:52.496550 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 02:57:52.496694 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 02:57:52.496991 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3513,"mutex_wait_us":28,"thread_start_us":3828,"threads_started":1}
I20260808 02:57:52.498541 11189 heartbeater.cc:347] Connected to a master server at 127.0.205.188:41233
I20260808 02:57:52.498700 11189 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:52.499074 11189 heartbeater.cc:511] Master 127.0.205.188:41233 requested a full tablet report, sending...
I20260808 02:57:52.500506 10760 ts_manager.cc:194] Registered new tserver with Master: be391462969e4dad920ea4aa06ee1169 (127.0.205.130:33955)
I20260808 02:57:52.588837 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 10150
I20260808 02:57:52.614817 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:41447
--local_ip_for_outbound_sockets=127.0.205.131
--tserver_master_addrs=127.0.205.190:34241
--webserver_port=35245
--webserver_interface=127.0.205.131
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab
--principal=oryx/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.205.190:34241,127.0.205.189:35803
--tserver_master_addrs=127.0.205.190:34241,127.0.205.189:35803,127.0.205.188:41233 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:57:52.844426 10996 heartbeater.cc:503] Master 127.0.205.190:34241 was elected leader, sending a full tablet report...
W20260808 02:57:52.958839 11205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:52.959175 11205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:52.959273 11205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:52.966765 11205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:52.966882 11205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:52.966995 11205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:52.976965 11205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab
--principal=oryx/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:41447
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=35245
--tserver_master_addrs=127.0.205.190:34241,127.0.205.189:35803,127.0.205.188:41233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:52.978343 11205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:52.979753 11205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:52.995013 11216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:52.994954 11214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:52.995390 11213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:52.995803 11205 server_base.cc:1034] running on GCE node
I20260808 02:57:52.996457 11205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:52.997506 11205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:52.998725 11205 hybrid_clock.cc:648] HybridClock initialized: now 1786157872998651 us; error 59 us; skew 500 ppm
I20260808 02:57:52.999076 11205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:52 dist-test-slave-6ntk krb5kdc[9660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:53.002638 11205 init.cc:377] Logged in from keytab as oryx/127.0.205.131@KRBTEST.COM (short username oryx)
I20260808 02:57:53.004114 11205 webserver.cc:460] Webserver started at http://127.0.205.131:35245/ using document root <none> and password file <none>
I20260808 02:57:53.004827 11205 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:53.004989 11205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:53.010618 11205 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:57:53.013396 11228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:53.014791 11205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:57:53.015002 11205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "e0e98ea59c2c4ec694d2b1c5899998d1"
format_stamp: "Formatted at 2026-08-08 02:57:49 on dist-test-slave-6ntk"
server_key: "a51a1933aa6955b6436e13b4798653bf"
server_key_iv: "cc8a0c47a86d89a1a436628f426812bd"
server_key_version: "encryptionkey@0"
I20260808 02:57:53.016048 11205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:53.045830 11205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:53.050580 11205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:53.050912 11205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:53.051991 11205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:57:53.053467 11205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:53.053596 11205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:53.053736 11205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:53.053828 11205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:53.089481 11205 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:41447
I20260808 02:57:53.089578 11391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:41447 every 8 connection(s)
I20260808 02:57:53.091082 11205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:57:53.092372 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 11205
Aug 08 02:57:53 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:57:53.139601 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.094598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:53301 (local address 127.0.205.190:34241)
0808 02:57:53.095916 (+ 1318us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.095925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.097717 (+ 1792us) server_negotiation.cc:407] Connection header received
0808 02:57:53.111339 (+ 13622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.111350 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.111492 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.111631 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:53.115117 (+ 3486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.116681 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.120550 (+ 3869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.121566 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.121739 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.129322 (+ 7583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:53.129334 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:53.129342 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:53.129448 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:53.134289 (+ 4841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.135634 (+ 1345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.135643 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.135649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.135847 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.136653 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.136659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.136662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.137131 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:53.137295 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.138868 (+ 1573us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.139151 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1133,"thread_start_us":982,"threads_started":1}
I20260808 02:57:53.142930 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.096751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.131:53301)
0808 02:57:53.097479 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:53.097505 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:53.110630 (+ 13125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:53.112281 (+ 1651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:53.112302 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:53.113645 (+ 1343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:53.114809 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.114846 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.116991 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.117001 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.120189 (+ 3188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.120212 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.122373 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.122385 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.122833 (+ 448us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.123877 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:53.123923 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:53.128895 (+ 4972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:53.134488 (+ 5593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:53.134498 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:53.134518 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:53.135374 (+ 856us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:53.136029 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:53.136037 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:53.136044 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:53.136505 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:53.137396 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:53.137405 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:53.138200 (+ 795us) client_negotiation.cc:769] Sending connection context
0808 02:57:53.139939 (+ 1739us) client_negotiation.cc:240] Negotiation successful
0808 02:57:53.140655 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":349,"threads_started":1}
I20260808 02:57:53.144910 11393 heartbeater.cc:347] Connected to a master server at 127.0.205.190:34241
I20260808 02:57:53.145285 11393 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:53.146420 11393 heartbeater.cc:511] Master 127.0.205.190:34241 requested a full tablet report, sending...
I20260808 02:57:53.148291 9722 ts_manager.cc:194] Re-registered known tserver with Master: e0e98ea59c2c4ec694d2b1c5899998d1 (127.0.205.131:41447)
I20260808 02:57:53.150085 9722 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:53301
Aug 08 02:57:53 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.189@KRBTEST.COM
Aug 08 02:57:53 dist-test-slave-6ntk krb5kdc[9660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157872, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.188@KRBTEST.COM
I20260808 02:57:53.170064 11409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.098088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:35803 (local address 127.0.205.131:51469)
0808 02:57:53.133822 (+ 35734us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:53.133852 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:53.134235 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:53.134903 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:53.134908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:53.135570 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:53.135950 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.135965 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.138789 (+ 2824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.138799 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.141341 (+ 2542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.141363 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.153643 (+ 12280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.153650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.153927 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.156400 (+ 2473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:53.156425 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:53.159650 (+ 3225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:53.166289 (+ 6639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:53.166294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:53.166298 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:53.166958 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:53.167589 (+ 631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:53.167598 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:53.167603 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:53.167813 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:53.168719 (+ 906us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:53.168727 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:53.168947 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 02:57:53.169109 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 02:57:53.169470 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35567,"thread_start_us":35794,"threads_started":1}
I20260808 02:57:53.171701 11415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.097723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:51469 (local address 127.0.205.189:35803)
0808 02:57:53.130520 (+ 32797us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.130528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.134369 (+ 3841us) server_negotiation.cc:407] Connection header received
0808 02:57:53.134457 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.134463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.134558 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.134682 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:53.136139 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.137532 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.141818 (+ 4286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.142703 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.142848 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.159880 (+ 17032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:53.159890 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:53.159896 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:53.159942 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:53.166109 (+ 6167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.167208 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.167214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.167218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.167415 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.167994 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.168001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.168007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.168540 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:53.170729 (+ 2189us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.170881 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.171138 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32623,"thread_start_us":40785,"threads_started":1}
I20260808 02:57:53.172972 11394 heartbeater.cc:347] Connected to a master server at 127.0.205.189:35803
I20260808 02:57:53.173169 11394 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:53.173820 11394 heartbeater.cc:511] Master 127.0.205.189:35803 requested a full tablet report, sending...
I20260808 02:57:53.174605 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.099968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37840 (local address 127.0.205.190:34241)
0808 02:57:53.127460 (+ 27492us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.127470 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.142175 (+ 14705us) server_negotiation.cc:407] Connection header received
0808 02:57:53.142371 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.142378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.142464 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.142580 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:53.153915 (+ 11335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.155664 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.157216 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.158059 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.158185 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.163200 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:53.163210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:53.163217 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:53.163273 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:53.168164 (+ 4891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.169382 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.169391 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.169394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.169596 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.170200 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.170208 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.170213 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.170720 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:53.171037 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.173624 (+ 2587us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.174030 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27275,"thread_start_us":35477,"threads_started":1}
I20260808 02:57:53.175251 10510 ts_manager.cc:194] Registered new tserver with Master: e0e98ea59c2c4ec694d2b1c5899998d1 (127.0.205.131:41447)
I20260808 02:57:53.181949 11399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.095673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.205.131:47371)
0808 02:57:53.096835 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:53.096893 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:53.126347 (+ 29454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:53.139242 (+ 12895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:53.139250 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:53.139809 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:53.140144 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.140156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.143123 (+ 2967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.143129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.144852 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.144869 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.148130 (+ 3261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.148137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.148350 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.152068 (+ 3718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:53.152094 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:53.166116 (+ 14022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:53.178053 (+ 11937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:53.178059 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:53.178063 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:53.178836 (+ 773us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:53.179378 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:53.179384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:53.179388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:53.179576 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:53.180500 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:53.180505 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:53.180742 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 02:57:53.180903 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 02:57:53.181372 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":534,"threads_started":1}
I20260808 02:57:53.183499 11400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.096363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:47371 (local address 127.0.205.188:41233)
0808 02:57:53.138604 (+ 42241us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.138612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.138635 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:57:53.138704 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.138709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.138817 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.138935 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:53.140354 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.141641 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.146918 (+ 5277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.147706 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.147841 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.166419 (+ 18578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:53.166429 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:53.166438 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:53.166493 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:53.172303 (+ 5810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.179034 (+ 6731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.179040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.179044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.179227 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.179717 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.179723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.179727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.180235 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:53.180415 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.182423 (+ 2008us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.183052 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42096,"thread_start_us":42012,"threads_started":1}
I20260808 02:57:53.184762 11392 heartbeater.cc:347] Connected to a master server at 127.0.205.188:41233
I20260808 02:57:53.184927 11392 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:53.185442 11392 heartbeater.cc:511] Master 127.0.205.188:41233 requested a full tablet report, sending...
I20260808 02:57:53.187018 10760 ts_manager.cc:194] Registered new tserver with Master: e0e98ea59c2c4ec694d2b1c5899998d1 (127.0.205.131:41447)
I20260808 02:57:53.192715 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.100738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39220 (local address 127.0.205.189:35803)
0808 02:57:53.139097 (+ 38359us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.139105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.145003 (+ 5898us) server_negotiation.cc:407] Connection header received
0808 02:57:53.145065 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.145070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.145158 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.145296 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:53.150253 (+ 4957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.151785 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.157856 (+ 6071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.158706 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.158853 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.164203 (+ 5350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:53.164212 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:53.164218 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:53.164268 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:53.177837 (+ 13569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.188978 (+ 11141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.188987 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.188993 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.189261 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.189952 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.189958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.189962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.190401 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:53.190715 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.191859 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.192156 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37246,"thread_start_us":29784,"threads_started":1}
I20260808 02:57:53.194842 11418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.142382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58422 (local address 127.0.205.188:41233)
0808 02:57:53.156500 (+ 14118us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.156510 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.173316 (+ 16806us) server_negotiation.cc:407] Connection header received
0808 02:57:53.173463 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.173470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.173566 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.173812 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:53.176543 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.178052 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.180502 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.181313 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.181423 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.186466 (+ 5043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:53.186474 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:53.186479 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:53.186518 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:53.190790 (+ 4272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.191614 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.191622 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.191625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.191807 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.192317 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.192324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.192328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.192746 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:53.193333 (+ 587us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.194135 (+ 802us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.194406 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13968,"thread_start_us":13794,"threads_started":1}
I20260808 02:57:53.180716 9722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:57:53.196784 9722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:53.243387 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.219846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42456 (local address 127.0.205.130:33955)
0808 02:57:53.220676 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.220686 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.232285 (+ 11599us) server_negotiation.cc:407] Connection header received
0808 02:57:53.232444 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.232457 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.233116 (+ 659us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.233532 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:53.235405 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.237434 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.240491 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.241628 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.241771 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.242060 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.242916 (+ 856us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.243094 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":394,"threads_started":1}
I20260808 02:57:53.243943 11438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.219446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:33955 (local address 127.0.0.1:42456)
0808 02:57:53.231788 (+ 12342us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:53.231810 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:53.232088 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:53.233532 (+ 1444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:53.233540 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:53.233582 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:53.235140 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.235157 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.237675 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.237680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.240272 (+ 2592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.240288 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.242137 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.242146 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.242374 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.242411 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:57:53.242510 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 02:57:53.242645 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12177,"mutex_wait_us":44,"thread_start_us":11615,"threads_started":1}
I20260808 02:57:53.244086 11435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.218722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:41447 (local address 127.0.0.1:42574)
0808 02:57:53.230712 (+ 11990us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:53.230740 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:53.230943 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:53.235469 (+ 4526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:53.235476 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:53.235503 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:53.235921 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.235933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.238488 (+ 2555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.238493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.240515 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.240532 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.243200 (+ 2668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.243207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.243368 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.243407 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 02:57:53.243505 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 02:57:53.243639 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11788,"mutex_wait_us":43,"thread_start_us":15272,"threads_started":1}
I20260808 02:57:53.246096 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.219199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.0.205.131:41447)
0808 02:57:53.234719 (+ 15520us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.234728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.234763 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:57:53.234866 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.234874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.235088 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.235475 (+ 387us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:53.236134 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.238296 (+ 2162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.240698 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.242499 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.242627 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.242821 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.243925 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.245715 (+ 1790us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15311,"thread_start_us":14990,"threads_started":1}
I20260808 02:57:53.247309 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.222004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:46555 (local address 127.0.0.1:52810)
0808 02:57:53.235276 (+ 13272us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:53.235297 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:53.235439 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:53.236654 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:53.236659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:53.236683 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:53.237148 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.237159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.239307 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.239313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.241667 (+ 2354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.241683 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.246493 (+ 4810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.246500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.246691 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.246723 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:57:53.246810 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 02:57:53.246941 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13091,"mutex_wait_us":38,"thread_start_us":9401,"threads_started":1}
I20260808 02:57:53.248273 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.222754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52810 (local address 127.0.205.129:46555)
0808 02:57:53.234870 (+ 12116us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.234876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.235670 (+ 794us) server_negotiation.cc:407] Connection header received
0808 02:57:53.235784 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.235793 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.236039 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.236392 (+ 353us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:53.237399 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.239143 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.244569 (+ 5426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.245990 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.246132 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.247729 (+ 1597us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.247803 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.247963 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11961,"thread_start_us":11839,"threads_started":1}
I20260808 02:57:53.248843 11101 tablet_service.cc:1467] Processing CreateTablet for tablet 5030f7bef3494a9faa096da4ffa6276a (DEFAULT_TABLE table=test-table [id=6220fcd21607434bb0f22382be14501b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:53.249727 11301 tablet_service.cc:1467] Processing CreateTablet for tablet 5030f7bef3494a9faa096da4ffa6276a (DEFAULT_TABLE table=test-table [id=6220fcd21607434bb0f22382be14501b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:53.250429 11101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5030f7bef3494a9faa096da4ffa6276a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:53.251050 11301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5030f7bef3494a9faa096da4ffa6276a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:53.252314 10907 tablet_service.cc:1467] Processing CreateTablet for tablet 5030f7bef3494a9faa096da4ffa6276a (DEFAULT_TABLE table=test-table [id=6220fcd21607434bb0f22382be14501b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:53.253654 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5030f7bef3494a9faa096da4ffa6276a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:53.263932 11441 tablet_bootstrap.cc:492] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: Bootstrap starting.
I20260808 02:57:53.267067 11442 tablet_bootstrap.cc:492] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: Bootstrap starting.
I20260808 02:57:53.268582 11441 tablet_bootstrap.cc:654] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:53.270364 11441 log.cc:824] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:53.270941 11442 tablet_bootstrap.cc:654] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:53.272598 11442 log.cc:824] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:53.275030 11443 tablet_bootstrap.cc:492] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: Bootstrap starting.
I20260808 02:57:53.276324 11442 tablet_bootstrap.cc:492] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: No bootstrap required, opened a new log
I20260808 02:57:53.276644 11442 ts_tablet_manager.cc:1397] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 02:57:53.276968 11441 tablet_bootstrap.cc:492] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: No bootstrap required, opened a new log
I20260808 02:57:53.277232 11441 ts_tablet_manager.cc:1397] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.010s
I20260808 02:57:53.278808 11443 tablet_bootstrap.cc:654] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:53.280118 11443 log.cc:824] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: Log is configured to *not* fsync() on all Append() calls
I20260808 02:57:53.285533 11441 raft_consensus.cc:348] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } } peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } }
I20260808 02:57:53.285995 11441 raft_consensus.cc:374] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:53.286110 11441 raft_consensus.cc:729] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e98ea59c2c4ec694d2b1c5899998d1, State: Initialized, Role: FOLLOWER
I20260808 02:57:53.286837 11441 consensus_queue.cc:260] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } } peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } }
I20260808 02:57:53.288671 11393 heartbeater.cc:503] Master 127.0.205.190:34241 was elected leader, sending a full tablet report...
I20260808 02:57:53.289214 11443 tablet_bootstrap.cc:492] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: No bootstrap required, opened a new log
I20260808 02:57:53.289219 11442 raft_consensus.cc:348] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } } peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } }
I20260808 02:57:53.289676 11443 ts_tablet_manager.cc:1397] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.003s
I20260808 02:57:53.289829 11442 raft_consensus.cc:374] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:53.289990 11442 raft_consensus.cc:729] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be391462969e4dad920ea4aa06ee1169, State: Initialized, Role: FOLLOWER
I20260808 02:57:53.290910 11441 ts_tablet_manager.cc:1428] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260808 02:57:53.290799 11442 consensus_queue.cc:260] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } } peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } }
I20260808 02:57:53.296869 11191 heartbeater.cc:503] Master 127.0.205.190:34241 was elected leader, sending a full tablet report...
I20260808 02:57:53.298353 11442 ts_tablet_manager.cc:1428] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: Time spent starting tablet: real 0.022s user 0.013s sys 0.010s
I20260808 02:57:53.305792 11443 raft_consensus.cc:348] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } } peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } }
I20260808 02:57:53.306403 11443 raft_consensus.cc:374] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:53.306547 11443 raft_consensus.cc:729] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cab275b91140e5bbcea6f6bd266e07, State: Initialized, Role: FOLLOWER
I20260808 02:57:53.307294 11443 consensus_queue.cc:260] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } } peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } }
I20260808 02:57:53.309314 11443 ts_tablet_manager.cc:1428] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: Time spent starting tablet: real 0.018s user 0.014s sys 0.000s
I20260808 02:57:53.319079 11448 raft_consensus.cc:482] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:53.319408 11448 raft_consensus.cc:504] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } } peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } }
I20260808 02:57:53.321287 11448 leader_election.cc:288] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0e98ea59c2c4ec694d2b1c5899998d1 (127.0.205.131:41447), 17cab275b91140e5bbcea6f6bd266e07 (127.0.205.129:46555)
I20260808 02:57:53.331496 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.321618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:39099 (local address 127.0.205.131:41447)
0808 02:57:53.321843 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.321851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.323161 (+ 1310us) server_negotiation.cc:407] Connection header received
0808 02:57:53.323401 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.323406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.323477 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.323570 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:53.324660 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.325833 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.328863 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.330240 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.330393 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.330471 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.331107 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.331251 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:57:53.331691 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.321708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:59321 (local address 127.0.205.129:46555)
0808 02:57:53.321957 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.321965 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.323160 (+ 1195us) server_negotiation.cc:407] Connection header received
0808 02:57:53.323384 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.323389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.323467 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.323570 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:53.324673 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.325795 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.328863 (+ 3068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.330218 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.330393 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.330471 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.331333 (+ 862us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.331475 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 02:57:53.332149 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.321661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:41447 (local address 127.0.205.130:39099)
0808 02:57:53.322923 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:53.322948 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:53.323213 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:53.323747 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:53.323752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:53.323791 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:53.324452 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.324465 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.326001 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.326005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.328660 (+ 2655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.328707 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.330614 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.330620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.330875 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.330930 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 02:57:53.331049 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 02:57:53.331197 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1106,"mutex_wait_us":206,"spinlock_wait_cycles":28672,"thread_start_us":836,"threads_started":1}
I20260808 02:57:53.332628 11451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.321719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:46555 (local address 127.0.205.130:59321)
0808 02:57:53.322956 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:53.322971 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:53.323213 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:53.323746 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:53.323752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:53.323788 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:53.324452 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.324465 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.325989 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.325994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.328679 (+ 2685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:53.328693 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.330587 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.330601 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:53.331151 (+ 550us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.331183 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:57:53.331895 (+ 712us) client_negotiation.cc:240] Negotiation successful
0808 02:57:53.332263 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"mutex_wait_us":398,"spinlock_wait_cycles":4096,"thread_start_us":763,"threads_started":1}
I20260808 02:57:53.333278 11329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5030f7bef3494a9faa096da4ffa6276a" candidate_uuid: "be391462969e4dad920ea4aa06ee1169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" is_pre_election: true
I20260808 02:57:53.333822 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5030f7bef3494a9faa096da4ffa6276a" candidate_uuid: "be391462969e4dad920ea4aa06ee1169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cab275b91140e5bbcea6f6bd266e07" is_pre_election: true
I20260808 02:57:53.333985 11329 raft_consensus.cc:2454] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be391462969e4dad920ea4aa06ee1169 in term 0.
I20260808 02:57:53.334408 10929 raft_consensus.cc:2454] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be391462969e4dad920ea4aa06ee1169 in term 0.
I20260808 02:57:53.334805 11044 leader_election.cc:302] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be391462969e4dad920ea4aa06ee1169, e0e98ea59c2c4ec694d2b1c5899998d1; no voters:
I20260808 02:57:53.335391 11448 raft_consensus.cc:2781] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:53.335638 11448 raft_consensus.cc:482] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:53.335772 11448 raft_consensus.cc:3037] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:53.338691 11448 raft_consensus.cc:504] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } } peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } }
I20260808 02:57:53.339393 11448 leader_election.cc:288] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [CANDIDATE]: Term 1 election: Requested vote from peers e0e98ea59c2c4ec694d2b1c5899998d1 (127.0.205.131:41447), 17cab275b91140e5bbcea6f6bd266e07 (127.0.205.129:46555)
I20260808 02:57:53.339820 11329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5030f7bef3494a9faa096da4ffa6276a" candidate_uuid: "be391462969e4dad920ea4aa06ee1169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1"
I20260808 02:57:53.340050 11329 raft_consensus.cc:3037] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:53.340034 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5030f7bef3494a9faa096da4ffa6276a" candidate_uuid: "be391462969e4dad920ea4aa06ee1169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cab275b91140e5bbcea6f6bd266e07"
I20260808 02:57:53.340296 10929 raft_consensus.cc:3037] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:53.342782 11329 raft_consensus.cc:2454] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be391462969e4dad920ea4aa06ee1169 in term 1.
I20260808 02:57:53.343281 11044 leader_election.cc:302] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be391462969e4dad920ea4aa06ee1169, e0e98ea59c2c4ec694d2b1c5899998d1; no voters:
I20260808 02:57:53.343621 11448 raft_consensus.cc:2781] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:53.343603 10929 raft_consensus.cc:2454] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be391462969e4dad920ea4aa06ee1169 in term 1.
I20260808 02:57:53.344275 11448 raft_consensus.cc:686] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 1 LEADER]: Becoming Leader. State: Replica: be391462969e4dad920ea4aa06ee1169, State: Running, Role: LEADER
I20260808 02:57:53.344871 11448 consensus_queue.cc:237] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } } peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } }
I20260808 02:57:53.351046 9722 catalog_manager.cc:5495] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 reported cstate change: term changed from 0 to 1, leader changed from <none> to be391462969e4dad920ea4aa06ee1169 (127.0.205.130). New cstate: current_term: 1 leader_uuid: "be391462969e4dad920ea4aa06ee1169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } health_report { overall_health: UNKNOWN } } }
W20260808 02:57:53.356998 11398 tablet.cc:2369] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:57:53.402879 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.396547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.0.205.130:33955)
0808 02:57:53.396784 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.396791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.396813 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:57:53.397004 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.397008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.397116 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.397307 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:53.397801 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.399033 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.400412 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.400882 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.401000 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.401600 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:53.402269 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:53.402398 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.402572 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.402647 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:57:53.411659 11329 raft_consensus.cc:1261] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1 [term 1 FOLLOWER]: Refusing update from remote peer be391462969e4dad920ea4aa06ee1169: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:53.411659 10929 raft_consensus.cc:1261] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07 [term 1 FOLLOWER]: Refusing update from remote peer be391462969e4dad920ea4aa06ee1169: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:53.412696 11452 consensus_queue.cc:1035] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:53.413069 11448 consensus_queue.cc:1035] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:53.427520 11455 mvcc.cc:205] Tried to move back new op lower bound from 7316102649483780096 to 7316102649224855552. Current Snapshot: MvccSnapshot[applied={T|T < 7316102649483780096}]
I20260808 02:57:53.452687 9719 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:53.453073 9719 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:53.460431 9719 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55a2aa7994d1497196b00b86336a13a1: Sending DeleteTablet for 3 replicas of tablet 5030f7bef3494a9faa096da4ffa6276a
I20260808 02:57:53.461499 11301 tablet_service.cc:1514] Processing DeleteTablet for tablet 5030f7bef3494a9faa096da4ffa6276a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:53 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:42574
I20260808 02:57:53.461755 11101 tablet_service.cc:1514] Processing DeleteTablet for tablet 5030f7bef3494a9faa096da4ffa6276a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:53 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:42456
I20260808 02:57:53.464569 10907 tablet_service.cc:1514] Processing DeleteTablet for tablet 5030f7bef3494a9faa096da4ffa6276a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:53 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:52810
I20260808 02:57:53.488152 11467 tablet_replica.cc:330] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: stopping tablet replica
I20260808 02:57:53.488636 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 9680
I20260808 02:57:53.490091 9761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:53.490568 11467 raft_consensus.cc:2229] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:57:53.491298 11470 tablet_replica.cc:330] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: stopping tablet replica
I20260808 02:57:53.493511 11470 raft_consensus.cc:2229] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:53.492915 11467 raft_consensus.cc:2258] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:53.495816 11470 raft_consensus.cc:2258] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:53.498415 11467 ts_tablet_manager.cc:1905] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:53.502254 11466 tablet_replica.cc:330] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: stopping tablet replica
I20260808 02:57:53.503041 11470 ts_tablet_manager.cc:1905] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:53.503634 11466 raft_consensus.cc:2229] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:53.504897 11466 raft_consensus.cc:2258] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:53.507333 11466 ts_tablet_manager.cc:1905] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:53.510861 11470 ts_tablet_manager.cc:1918] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:53.511051 11470 log.cc:1196] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/5030f7bef3494a9faa096da4ffa6276a
I20260808 02:57:53.511684 11470 ts_tablet_manager.cc:1939] T 5030f7bef3494a9faa096da4ffa6276a P 17cab275b91140e5bbcea6f6bd266e07: Deleting consensus metadata
I20260808 02:57:53.515125 11466 ts_tablet_manager.cc:1918] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:53.515380 11466 log.cc:1196] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/5030f7bef3494a9faa096da4ffa6276a
I20260808 02:57:53.516022 11466 ts_tablet_manager.cc:1939] T 5030f7bef3494a9faa096da4ffa6276a P e0e98ea59c2c4ec694d2b1c5899998d1: Deleting consensus metadata
I20260808 02:57:53.517347 11467 ts_tablet_manager.cc:1918] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:53.517630 11467 log.cc:1196] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/5030f7bef3494a9faa096da4ffa6276a
I20260808 02:57:53.518410 11467 ts_tablet_manager.cc:1939] T 5030f7bef3494a9faa096da4ffa6276a P be391462969e4dad920ea4aa06ee1169: Deleting consensus metadata
I20260808 02:57:53.534076 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.474032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39226 (local address 127.0.205.189:35803)
0808 02:57:53.474223 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.474229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.486527 (+ 12298us) server_negotiation.cc:407] Connection header received
0808 02:57:53.488929 (+ 2402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.488935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.489024 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.489103 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:53.499936 (+ 10833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.501123 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.505217 (+ 4094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.506069 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.506201 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.515569 (+ 9368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:53.515580 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:53.515587 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:53.515647 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:53.519866 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.521048 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.521055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.521059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.521231 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.521732 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.521738 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.521742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.522179 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:53.522442 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.522842 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.523089 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:57:53.541383 11418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.487217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.0.205.188:41233)
0808 02:57:53.487449 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.487458 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.523496 (+ 36038us) server_negotiation.cc:407] Connection header received
0808 02:57:53.523857 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.523864 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.523982 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.524058 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:53.526235 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.527334 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.529343 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.530060 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.530191 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.534055 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:53.534062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:53.534068 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:53.534113 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:53.537936 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.538835 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.538860 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.538864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.539034 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.539559 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.539564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.539568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.539923 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:53.540164 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.540773 (+ 609us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.541023 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:57:53.647888 9702 catalog_manager.cc:4841] TS be391462969e4dad920ea4aa06ee1169 (127.0.205.130:33955): tablet 5030f7bef3494a9faa096da4ffa6276a (table test-table [id=6220fcd21607434bb0f22382be14501b]) successfully deleted
I20260808 02:57:53.648029 9701 catalog_manager.cc:4841] TS 17cab275b91140e5bbcea6f6bd266e07 (127.0.205.129:46555): tablet 5030f7bef3494a9faa096da4ffa6276a (table test-table [id=6220fcd21607434bb0f22382be14501b]) successfully deleted
I20260808 02:57:53.648411 9707 catalog_manager.cc:4841] TS e0e98ea59c2c4ec694d2b1c5899998d1 (127.0.205.131:41447): tablet 5030f7bef3494a9faa096da4ffa6276a (table test-table [id=6220fcd21607434bb0f22382be14501b]) successfully deleted
I20260808 02:57:53.648983 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 9680
I20260808 02:57:53.711632 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.695223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39238 (local address 127.0.205.189:35803)
0808 02:57:53.695419 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.695428 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.695450 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:57:53.695703 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.695711 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.695816 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.695919 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:53.698403 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.699569 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.701136 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.701573 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.701752 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.704988 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:53.704993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:53.704996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:53.705031 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:53.708553 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.709367 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.709376 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.709381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.709557 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.710045 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.710050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.710053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.710372 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:53.710665 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.711018 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.711247 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:57:53.713303 11418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:53.696353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58430 (local address 127.0.205.188:41233)
0808 02:57:53.696575 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:53.696582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:53.696692 (+ 110us) server_negotiation.cc:407] Connection header received
0808 02:57:53.696891 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:53.696897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:53.696984 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:53.697095 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:53.699207 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.700150 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.702124 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:53.702716 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:53.702879 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:53.706260 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:53.706267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:53.706274 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:53.706322 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:53.709464 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.710528 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.710534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.710537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.710690 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:53.711118 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:53.711123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:53.711127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:53.711484 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:53.711685 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:53.712738 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0808 02:57:53.712969 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:57:54.443609 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.442753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.130:51043)
0808 02:57:54.443312 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.443449 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":279,"threads_started":1}
W20260808 02:57:54.444199 11191 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:34241 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
I20260808 02:57:54.446301 11485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.445075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.131:36631)
0808 02:57:54.445563 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.445728 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":255,"threads_started":1}
W20260808 02:57:54.447638 11393 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:34241 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
I20260808 02:57:54.453409 11486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.447293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.129:59457)
0808 02:57:54.453109 (+ 5816us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.453248 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5679,"thread_start_us":5586,"threads_started":1}
W20260808 02:57:54.454025 10996 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:34241 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
I20260808 02:57:54.961431 11487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 55a2aa7994d1497196b00b86336a13a1)
I20260808 02:57:54.961681 11487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } }
I20260808 02:57:54.962515 11487 leader_election.cc:288] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 55a2aa7994d1497196b00b86336a13a1 (127.0.205.190:34241), e57eb3ef7a1b4878a6c6627ddfc960e4 (127.0.205.189:35803)
I20260808 02:57:54.965317 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.962766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38254)
0808 02:57:54.964649 (+ 1883us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.964993 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1675,"thread_start_us":1472,"threads_started":1}
W20260808 02:57:54.965668 10740 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
I20260808 02:57:54.966912 11493 raft_consensus.cc:482] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 55a2aa7994d1497196b00b86336a13a1)
I20260808 02:57:54.967136 11493 raft_consensus.cc:504] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } }
I20260808 02:57:54.967556 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.967150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38264)
0808 02:57:54.967277 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.967400 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 02:57:54.967773 10740 leader_election.cc:334] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 55a2aa7994d1497196b00b86336a13a1 (127.0.205.190:34241): Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
I20260808 02:57:54.968268 11493 leader_election.cc:288] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 55a2aa7994d1497196b00b86336a13a1 (127.0.205.190:34241), 9f21bbc67b8949eda7ab74bb20f71eba (127.0.205.188:41233)
I20260808 02:57:54.973963 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.962866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:35803 (local address 127.0.0.1:39254)
0808 02:57:54.964946 (+ 2080us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.964973 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:54.965140 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:54.965659 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:54.965662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:54.965686 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:54.966098 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:54.966112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:54.967802 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:54.967808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:54.970461 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:54.970484 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:54.972831 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:54.972839 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:54.973096 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:54.973178 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 02:57:54.973264 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:57:54.973471 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1894,"thread_start_us":1718,"threads_started":1}
I20260808 02:57:54.975268 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.962928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39254 (local address 127.0.205.189:35803)
0808 02:57:54.965179 (+ 2251us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:54.965187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:54.965210 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:57:54.965381 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:54.965386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:54.965498 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:54.965622 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:54.966356 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:54.967656 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:54.970704 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:54.972557 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:54.974322 (+ 1765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:54.974484 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:54.974552 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 02:57:54.974744 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2127,"thread_start_us":1954,"threads_started":1}
I20260808 02:57:54.979152 10521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" is_pre_election: true
I20260808 02:57:54.979347 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.968382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38274)
0808 02:57:54.978729 (+ 10347us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.979040 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10188,"thread_start_us":10018,"threads_started":1}
I20260808 02:57:54.979498 10521 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f21bbc67b8949eda7ab74bb20f71eba in term 1.
W20260808 02:57:54.979662 10491 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
I20260808 02:57:54.980665 10741 leader_election.cc:302] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f21bbc67b8949eda7ab74bb20f71eba, e57eb3ef7a1b4878a6c6627ddfc960e4; no voters: 55a2aa7994d1497196b00b86336a13a1
I20260808 02:57:54.981063 11487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 02:57:54.981210 11487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Starting leader election (detected failure of leader 55a2aa7994d1497196b00b86336a13a1)
I20260808 02:57:54.981318 11487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:57:54.981794 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.981341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38278)
0808 02:57:54.981505 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.981567 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 02:57:54.982031 10491 leader_election.cc:334] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 55a2aa7994d1497196b00b86336a13a1 (127.0.205.190:34241): Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
I20260808 02:57:54.984534 11487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } }
I20260808 02:57:54.985242 11487 leader_election.cc:288] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [CANDIDATE]: Term 2 election: Requested vote from peers 55a2aa7994d1497196b00b86336a13a1 (127.0.205.190:34241), e57eb3ef7a1b4878a6c6627ddfc960e4 (127.0.205.189:35803)
I20260808 02:57:54.985638 10521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4"
I20260808 02:57:54.985858 10521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:57:54.986012 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.985388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38294)
0808 02:57:54.985663 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.985775 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260808 02:57:54.987566 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.968895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.0.205.188:41233)
0808 02:57:54.978438 (+ 9543us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:54.978444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:54.979042 (+ 598us) server_negotiation.cc:407] Connection header received
0808 02:57:54.979239 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:54.979244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:54.979312 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:54.979428 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:54.980587 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:54.981900 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:54.984306 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:54.985800 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:54.986025 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:54.986201 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:54.987074 (+ 873us) server_negotiation.cc:299] Negotiation successful
0808 02:57:54.987217 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9370,"thread_start_us":9211,"threads_started":1}
I20260808 02:57:54.987960 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.968444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:41233 (local address 127.0.0.1:58442)
0808 02:57:54.978920 (+ 10476us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.978949 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:54.979089 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:54.979858 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:54.979862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:54.979888 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:54.980339 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:54.980360 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:54.982068 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:54.982074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:54.984108 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:54.984124 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:54.986516 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:54.986527 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:54.986818 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:54.986889 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 02:57:54.987048 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 02:57:54.987257 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10338,"thread_start_us":10125,"threads_started":1}
I20260808 02:57:54.988745 10770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" is_pre_election: true
I20260808 02:57:54.989099 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:54.988574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38302)
0808 02:57:54.988748 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:54.988861 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 02:57:54.989180 10770 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e57eb3ef7a1b4878a6c6627ddfc960e4 in current term 2: Already voted for candidate 9f21bbc67b8949eda7ab74bb20f71eba in this term.
W20260808 02:57:54.989364 10740 leader_election.cc:334] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 55a2aa7994d1497196b00b86336a13a1 (127.0.205.190:34241): Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
I20260808 02:57:54.989836 10492 leader_election.cc:302] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e57eb3ef7a1b4878a6c6627ddfc960e4; no voters: 55a2aa7994d1497196b00b86336a13a1, 9f21bbc67b8949eda7ab74bb20f71eba
I20260808 02:57:54.990163 10521 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f21bbc67b8949eda7ab74bb20f71eba in term 2.
I20260808 02:57:54.990386 11493 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:57:54.990674 10741 leader_election.cc:302] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f21bbc67b8949eda7ab74bb20f71eba, e57eb3ef7a1b4878a6c6627ddfc960e4; no voters: 55a2aa7994d1497196b00b86336a13a1
I20260808 02:57:54.991001 11487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:57:54.991463 11487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [term 2 LEADER]: Becoming Leader. State: Replica: 9f21bbc67b8949eda7ab74bb20f71eba, State: Running, Role: LEADER
I20260808 02:57:54.991848 11487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } }
I20260808 02:57:54.994410 11502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f21bbc67b8949eda7ab74bb20f71eba. Latest consensus state: current_term: 2 leader_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:54.994840 11502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:54.995517 11505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:54.999532 11505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:55.000442 11505 catalog_manager.cc:1259] Loaded cluster ID: 12460217a9d5465098796d959476dd8e
I20260808 02:57:55.000516 11505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:55.001657 11505 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:55.002481 11505 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Loaded TSK: 0
I20260808 02:57:55.003376 11505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:57:55.056658 10759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:57:55.064010 10759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:57:55.068670 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.068232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38324)
0808 02:57:55.068429 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.068535 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 02:57:55.069044 10521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [term 2 FOLLOWER]: Refusing update from remote peer 9f21bbc67b8949eda7ab74bb20f71eba: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 02:57:55.069823 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.069326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38338)
0808 02:57:55.069550 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.069642 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 02:57:55.069927 11502 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [LEADER]: Connected to new peer: Peer: permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 02:57:55.070194 10740 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba -> Peer 55a2aa7994d1497196b00b86336a13a1 (127.0.205.190:34241): Couldn't send request to peer 55a2aa7994d1497196b00b86336a13a1. Status: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:57:55.072765 11493 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f21bbc67b8949eda7ab74bb20f71eba. Latest consensus state: current_term: 2 leader_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:55.073052 11493 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:57:55.075073 11493 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:55.075392 11493 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57eb3ef7a1b4878a6c6627ddfc960e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:57:55.075984 11502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "55a2aa7994d1497196b00b86336a13a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34241 } } peers { permanent_uuid: "e57eb3ef7a1b4878a6c6627ddfc960e4" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 35803 } attrs { promote: false } } peers { permanent_uuid: "9f21bbc67b8949eda7ab74bb20f71eba" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 41233 } attrs { promote: false } } }
I20260808 02:57:55.076260 11502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:55.092674 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.084679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:46555 (local address 127.0.0.1:52814)
0808 02:57:55.084978 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.084996 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:55.085090 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:55.086187 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:55.086192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:55.086218 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:55.086530 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:55.086543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.088000 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.088006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:55.090124 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:55.090136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.091837 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.091843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:55.092035 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.092083 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 02:57:55.092149 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 02:57:55.092292 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":28}
I20260808 02:57:55.092700 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.083951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:33955 (local address 127.0.0.1:42474)
0808 02:57:55.084191 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.084212 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:55.084322 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:55.084991 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:55.084996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:55.085029 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:55.085266 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:55.085278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.087590 (+ 2312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.087595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:55.089574 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:55.089589 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.092044 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.092049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:55.092232 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.092261 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:57:55.092312 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 02:57:55.092426 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":1}
I20260808 02:57:55.093557 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.084083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42474 (local address 127.0.205.130:33955)
0808 02:57:55.084598 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:55.084604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:55.084622 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:57:55.084699 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:55.084703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:55.084778 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:55.084964 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:55.086268 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.087433 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.090562 (+ 3129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.091870 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.092985 (+ 1115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.093086 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:55.093136 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 02:57:55.093272 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":270,"threads_started":1}
I20260808 02:57:55.093573 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.084785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52814 (local address 127.0.205.129:46555)
0808 02:57:55.085777 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:55.085786 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:55.085809 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:57:55.085919 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:55.085924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:55.086007 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:55.086744 (+ 737us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:55.086911 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.087866 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.090283 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.091594 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.092931 (+ 1337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.093037 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:55.093094 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 02:57:55.093233 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":722,"threads_started":1}
I20260808 02:57:55.094151 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.085468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:41447 (local address 127.0.0.1:42582)
0808 02:57:55.086139 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.086159 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:55.086287 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:55.086801 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:55.086805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:55.086831 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:55.087172 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:55.087183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.088408 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.088414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:55.090688 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:55.090703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.092591 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.092597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:55.093278 (+ 681us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.093321 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:57:55.093421 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 02:57:55.093549 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":30,"thread_start_us":311,"threads_started":1}
I20260808 02:57:55.094491 11101 tablet_service.cc:1467] Processing CreateTablet for tablet 468334945e614daa817c314acadd707d (DEFAULT_TABLE table=test-table [id=d2d4f0ee401d4b3aaa6b938a6832e9fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:55.094491 10907 tablet_service.cc:1467] Processing CreateTablet for tablet 468334945e614daa817c314acadd707d (DEFAULT_TABLE table=test-table [id=d2d4f0ee401d4b3aaa6b938a6832e9fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:55.094980 10907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 468334945e614daa817c314acadd707d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:55.094980 11101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 468334945e614daa817c314acadd707d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:55.095100 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.085721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.0.205.131:41447)
0808 02:57:55.086336 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:55.086343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:55.086365 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:57:55.086441 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:55.086447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:55.086637 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:55.086745 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:55.087341 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.088278 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.090872 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.092195 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.092313 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.092436 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:55.094657 (+ 2221us) server_negotiation.cc:299] Negotiation successful
0808 02:57:55.094805 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":365,"threads_started":1}
I20260808 02:57:55.095784 11301 tablet_service.cc:1467] Processing CreateTablet for tablet 468334945e614daa817c314acadd707d (DEFAULT_TABLE table=test-table [id=d2d4f0ee401d4b3aaa6b938a6832e9fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:57:55.096124 11301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 468334945e614daa817c314acadd707d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:55.103236 11518 tablet_bootstrap.cc:492] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: Bootstrap starting.
I20260808 02:57:55.104007 11519 tablet_bootstrap.cc:492] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: Bootstrap starting.
I20260808 02:57:55.106669 11519 tablet_bootstrap.cc:654] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:55.107686 11518 tablet_bootstrap.cc:654] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:55.108348 11517 tablet_bootstrap.cc:492] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1: Bootstrap starting.
I20260808 02:57:55.109836 11517 tablet_bootstrap.cc:654] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:55.110402 11519 tablet_bootstrap.cc:492] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: No bootstrap required, opened a new log
I20260808 02:57:55.110576 11519 ts_tablet_manager.cc:1397] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 02:57:55.111240 11519 raft_consensus.cc:348] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } } peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } }
I20260808 02:57:55.111478 11519 raft_consensus.cc:374] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:55.111589 11519 raft_consensus.cc:729] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cab275b91140e5bbcea6f6bd266e07, State: Initialized, Role: FOLLOWER
I20260808 02:57:55.111861 11519 consensus_queue.cc:260] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } } peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } }
I20260808 02:57:55.111977 11517 tablet_bootstrap.cc:492] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1: No bootstrap required, opened a new log
I20260808 02:57:55.112134 11517 ts_tablet_manager.cc:1397] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 02:57:55.113409 11518 tablet_bootstrap.cc:492] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: No bootstrap required, opened a new log
I20260808 02:57:55.113634 11518 ts_tablet_manager.cc:1397] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260808 02:57:55.114562 11518 raft_consensus.cc:348] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } } peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } }
I20260808 02:57:55.114867 11518 raft_consensus.cc:374] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:55.114989 11518 raft_consensus.cc:729] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be391462969e4dad920ea4aa06ee1169, State: Initialized, Role: FOLLOWER
I20260808 02:57:55.115224 11518 consensus_queue.cc:260] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } } peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } }
I20260808 02:57:55.115988 11519 ts_tablet_manager.cc:1428] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260808 02:57:55.116487 11518 ts_tablet_manager.cc:1428] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:57:55.117565 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.112972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.129:41817)
0808 02:57:55.117068 (+ 4096us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.117256 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3961,"thread_start_us":3808,"threads_started":1}
I20260808 02:57:55.117537 10995 heartbeater.cc:503] Master 127.0.205.188:41233 was elected leader, sending a full tablet report...
I20260808 02:57:55.118901 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.118453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.129:32855)
0808 02:57:55.118649 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.118749 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 02:57:55.118640 11517 raft_consensus.cc:348] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } } peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } }
I20260808 02:57:55.119485 11517 raft_consensus.cc:374] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:55.119679 11517 raft_consensus.cc:729] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e98ea59c2c4ec694d2b1c5899998d1, State: Initialized, Role: FOLLOWER
I20260808 02:57:55.119967 11517 consensus_queue.cc:260] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } } peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } }
I20260808 02:57:55.120942 11531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.119067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.130:39403)
0808 02:57:55.120407 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.120585 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1188,"thread_start_us":894,"threads_started":1}
I20260808 02:57:55.122476 11517 ts_tablet_manager.cc:1428] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1: Time spent starting tablet: real 0.010s user 0.000s sys 0.005s
I20260808 02:57:55.123562 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.122717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.131:59529)
0808 02:57:55.123268 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.123371 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":295,"threads_started":1}
I20260808 02:57:55.124303 11392 heartbeater.cc:503] Master 127.0.205.188:41233 was elected leader, sending a full tablet report...
I20260808 02:57:55.124686 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.124255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.131:49467)
0808 02:57:55.124431 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.124519 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 02:57:55.200359 11193 tablet.cc:2369] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:57:55.290895 11532 raft_consensus.cc:482] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:57:55.291208 11532 raft_consensus.cc:504] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } } peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } }
I20260808 02:57:55.292876 11532 leader_election.cc:288] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be391462969e4dad920ea4aa06ee1169 (127.0.205.130:33955), 17cab275b91140e5bbcea6f6bd266e07 (127.0.205.129:46555)
I20260808 02:57:55.301779 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.293332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:41285 (local address 127.0.205.129:46555)
0808 02:57:55.293615 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:55.293625 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:55.294151 (+ 526us) server_negotiation.cc:407] Connection header received
0808 02:57:55.294264 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:55.294271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:55.294342 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:55.294432 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:55.295064 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.296116 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.299102 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.300497 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.300680 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.300761 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:55.301358 (+ 597us) server_negotiation.cc:299] Negotiation successful
0808 02:57:55.301521 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 02:57:55.301816 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.293344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:53501 (local address 127.0.205.130:33955)
0808 02:57:55.293550 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:55.293557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:55.293677 (+ 120us) server_negotiation.cc:407] Connection header received
0808 02:57:55.294055 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:55.294060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:55.294174 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:55.294287 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:57:55.294936 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.296054 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.299101 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.300459 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.300680 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.300761 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:55.301426 (+ 665us) server_negotiation.cc:299] Negotiation successful
0808 02:57:55.301564 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:57:55.301949 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.293200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:46555 (local address 127.0.205.131:41285)
0808 02:57:55.294056 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.294070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:55.294163 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:55.294570 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:55.294574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:55.294594 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:55.294924 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:55.294932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.296251 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.296257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:55.298939 (+ 2682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:55.298948 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.300786 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.300793 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:55.301066 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.301192 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 02:57:55.301313 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 02:57:55.301471 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":367,"thread_start_us":469,"threads_started":1}
I20260808 02:57:55.302542 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.293200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:33955 (local address 127.0.205.131:53501)
0808 02:57:55.293547 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.293576 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:55.293851 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:55.294320 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:55.294324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:55.294352 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:57:55.294763 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:55.294774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.296258 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.296262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:55.298911 (+ 2649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:55.298934 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.300801 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.300808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:55.301116 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.301189 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 02:57:55.302145 (+ 956us) client_negotiation.cc:240] Negotiation successful
0808 02:57:55.302266 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":251}
I20260808 02:57:55.303105 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "468334945e614daa817c314acadd707d" candidate_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cab275b91140e5bbcea6f6bd266e07" is_pre_election: true
I20260808 02:57:55.303412 10929 raft_consensus.cc:2454] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0e98ea59c2c4ec694d2b1c5899998d1 in term 0.
I20260808 02:57:55.303642 11132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "468334945e614daa817c314acadd707d" candidate_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be391462969e4dad920ea4aa06ee1169" is_pre_election: true
I20260808 02:57:55.304040 11229 leader_election.cc:302] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cab275b91140e5bbcea6f6bd266e07, e0e98ea59c2c4ec694d2b1c5899998d1; no voters:
I20260808 02:57:55.304172 11132 raft_consensus.cc:2454] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0e98ea59c2c4ec694d2b1c5899998d1 in term 0.
I20260808 02:57:55.304517 11532 raft_consensus.cc:2781] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:57:55.304682 11532 raft_consensus.cc:482] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:57:55.304786 11532 raft_consensus.cc:3037] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 0 FOLLOWER]: Advancing to term 1
W20260808 02:57:55.306257 10998 tablet.cc:2369] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:57:55.307979 11532 raft_consensus.cc:504] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } } peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } }
I20260808 02:57:55.308669 11532 leader_election.cc:288] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [CANDIDATE]: Term 1 election: Requested vote from peers be391462969e4dad920ea4aa06ee1169 (127.0.205.130:33955), 17cab275b91140e5bbcea6f6bd266e07 (127.0.205.129:46555)
I20260808 02:57:55.309201 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "468334945e614daa817c314acadd707d" candidate_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cab275b91140e5bbcea6f6bd266e07"
I20260808 02:57:55.309428 10929 raft_consensus.cc:3037] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:55.312600 10929 raft_consensus.cc:2454] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e98ea59c2c4ec694d2b1c5899998d1 in term 1.
I20260808 02:57:55.313035 11229 leader_election.cc:302] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cab275b91140e5bbcea6f6bd266e07, e0e98ea59c2c4ec694d2b1c5899998d1; no voters:
I20260808 02:57:55.313026 11132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "468334945e614daa817c314acadd707d" candidate_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be391462969e4dad920ea4aa06ee1169"
I20260808 02:57:55.313256 11132 raft_consensus.cc:3037] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:55.313351 11532 raft_consensus.cc:2781] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:55.315541 11132 raft_consensus.cc:2454] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0e98ea59c2c4ec694d2b1c5899998d1 in term 1.
I20260808 02:57:55.316325 11532 raft_consensus.cc:686] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 1 LEADER]: Becoming Leader. State: Replica: e0e98ea59c2c4ec694d2b1c5899998d1, State: Running, Role: LEADER
I20260808 02:57:55.317029 11532 consensus_queue.cc:237] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } } peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } }
I20260808 02:57:55.317443 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.316966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.131:51609)
0808 02:57:55.317142 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.317253 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 02:57:55.320892 10759 catalog_manager.cc:5495] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0e98ea59c2c4ec694d2b1c5899998d1 (127.0.205.131). New cstate: current_term: 1 leader_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0e98ea59c2c4ec694d2b1c5899998d1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41447 } health_report { overall_health: HEALTHY } } }
I20260808 02:57:55.340493 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.334935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42596 (local address 127.0.205.131:41447)
0808 02:57:55.335139 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:55.335143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:55.335161 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:57:55.335358 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:55.335362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:55.335449 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:55.335575 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:57:55.335877 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.336613 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.338259 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.338819 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.338925 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.339287 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:57:55.339879 (+ 592us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:57:55.339999 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:55.340093 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 02:57:55.340165 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 02:57:55.344638 10929 raft_consensus.cc:1261] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07 [term 1 FOLLOWER]: Refusing update from remote peer e0e98ea59c2c4ec694d2b1c5899998d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:55.345001 11132 raft_consensus.cc:1261] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169 [term 1 FOLLOWER]: Refusing update from remote peer e0e98ea59c2c4ec694d2b1c5899998d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:57:55.345435 11540 consensus_queue.cc:1035] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17cab275b91140e5bbcea6f6bd266e07" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:55.346242 11532 consensus_queue.cc:1035] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be391462969e4dad920ea4aa06ee1169" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:57:55.348523 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.347811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.129:55617)
0808 02:57:55.348082 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.348228 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260808 02:57:55.348826 11189 heartbeater.cc:503] Master 127.0.205.188:41233 was elected leader, sending a full tablet report...
I20260808 02:57:55.349998 11531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.349334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.130:56397)
0808 02:57:55.349541 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.349669 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 02:57:55.354595 11546 mvcc.cc:205] Tried to move back new op lower bound from 7316102657404547072 to 7316102657303105536. Current Snapshot: MvccSnapshot[applied={T|T < 7316102657404547072}]
I20260808 02:57:55.355881 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.354738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.131:49915)
0808 02:57:55.355540 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.355662 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":630}
I20260808 02:57:55.357363 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.356707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.129:46977)
0808 02:57:55.356995 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.357109 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260808 02:57:55.360822 11549 mvcc.cc:205] Tried to move back new op lower bound from 7316102657404547072 to 7316102657303105536. Current Snapshot: MvccSnapshot[applied={T|T < 7316102657404547072}]
I20260808 02:57:55.369057 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.368281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.131:46379)
0808 02:57:55.368729 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.368835 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326}
I20260808 02:57:55.371534 11531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.370869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.205.130:55723)
0808 02:57:55.371148 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.371290 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260808 02:57:55.384111 10759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:55.384382 10759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:57:55.390542 10759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f21bbc67b8949eda7ab74bb20f71eba: Sending DeleteTablet for 3 replicas of tablet 468334945e614daa817c314acadd707d
I20260808 02:57:55.391721 11101 tablet_service.cc:1514] Processing DeleteTablet for tablet 468334945e614daa817c314acadd707d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:55 UTC) from {username='oryx', principal='oryx/127.0.205.188@KRBTEST.COM'} at 127.0.0.1:42474
I20260808 02:57:55.391945 10907 tablet_service.cc:1514] Processing DeleteTablet for tablet 468334945e614daa817c314acadd707d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:55 UTC) from {username='oryx', principal='oryx/127.0.205.188@KRBTEST.COM'} at 127.0.0.1:52814
I20260808 02:57:55.392450 11301 tablet_service.cc:1514] Processing DeleteTablet for tablet 468334945e614daa817c314acadd707d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:57:55 UTC) from {username='oryx', principal='oryx/127.0.205.188@KRBTEST.COM'} at 127.0.0.1:42582
I20260808 02:57:55.394822 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 10730
I20260808 02:57:55.401324 11563 tablet_replica.cc:330] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: stopping tablet replica
I20260808 02:57:55.402244 11563 raft_consensus.cc:2229] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:55.402760 11563 raft_consensus.cc:2258] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:55.404320 11563 ts_tablet_manager.cc:1905] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:55.411319 11564 tablet_replica.cc:330] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1: stopping tablet replica
I20260808 02:57:55.411644 11564 raft_consensus.cc:2229] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:57:55.412035 11562 tablet_replica.cc:330] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: stopping tablet replica
I20260808 02:57:55.412256 11564 raft_consensus.cc:2258] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:55.412315 11562 raft_consensus.cc:2229] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:57:55.412643 11562 raft_consensus.cc:2258] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:57:55.413090 11563 ts_tablet_manager.cc:1918] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:55.413254 11563 log.cc:1196] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/468334945e614daa817c314acadd707d
I20260808 02:57:55.413589 11564 ts_tablet_manager.cc:1905] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:55.413767 11563 ts_tablet_manager.cc:1939] T 468334945e614daa817c314acadd707d P 17cab275b91140e5bbcea6f6bd266e07: Deleting consensus metadata
I20260808 02:57:55.414763 11562 ts_tablet_manager.cc:1905] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:57:55.415158 10739 catalog_manager.cc:4841] TS 17cab275b91140e5bbcea6f6bd266e07 (127.0.205.129:46555): tablet 468334945e614daa817c314acadd707d (table test-table [id=d2d4f0ee401d4b3aaa6b938a6832e9fc]) successfully deleted
I20260808 02:57:55.421324 11564 ts_tablet_manager.cc:1918] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:55.421516 11564 log.cc:1196] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/468334945e614daa817c314acadd707d
I20260808 02:57:55.421851 11562 ts_tablet_manager.cc:1918] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:57:55.422077 11562 log.cc:1196] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/468334945e614daa817c314acadd707d
I20260808 02:57:55.422135 11564 ts_tablet_manager.cc:1939] T 468334945e614daa817c314acadd707d P e0e98ea59c2c4ec694d2b1c5899998d1: Deleting consensus metadata
I20260808 02:57:55.422667 11562 ts_tablet_manager.cc:1939] T 468334945e614daa817c314acadd707d P be391462969e4dad920ea4aa06ee1169: Deleting consensus metadata
I20260808 02:57:55.423518 10742 catalog_manager.cc:4841] TS e0e98ea59c2c4ec694d2b1c5899998d1 (127.0.205.131:41447): tablet 468334945e614daa817c314acadd707d (table test-table [id=d2d4f0ee401d4b3aaa6b938a6832e9fc]) successfully deleted
I20260808 02:57:55.423926 10740 catalog_manager.cc:4841] TS be391462969e4dad920ea4aa06ee1169 (127.0.205.130:33955): tablet 468334945e614daa817c314acadd707d (table test-table [id=d2d4f0ee401d4b3aaa6b938a6832e9fc]) successfully deleted
I20260808 02:57:55.429575 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.396230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52820 (local address 127.0.205.129:46555)
0808 02:57:55.396494 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:55.396503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:55.402399 (+ 5896us) server_negotiation.cc:407] Connection header received
0808 02:57:55.402727 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:55.402736 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:55.402846 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:55.402931 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:55.405087 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.406313 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.418055 (+ 11742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.418760 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.418918 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.423049 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:55.423073 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:55.423095 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:55.423143 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:55.426567 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:55.427512 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:55.427520 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:55.427523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:55.427661 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:55.428071 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:55.428075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:55.428077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:55.428458 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:55.428699 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:55.429056 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 02:57:55.429226 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:57:55.430493 10988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:55.549031 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 10730
I20260808 02:57:55.582212 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.581534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38344)
0808 02:57:55.581860 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.582010 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260808 02:57:55.582258 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 11012
I20260808 02:57:55.584197 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.583676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38350)
0808 02:57:55.583965 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:55.584065 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 02:57:55.600220 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.583234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42482 (local address 127.0.205.130:33955)
0808 02:57:55.583418 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:55.583424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:55.583558 (+ 134us) server_negotiation.cc:407] Connection header received
0808 02:57:55.583766 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:55.583773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:55.583862 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:55.583977 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:55.586102 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.587209 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.588657 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.589272 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.589419 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.592061 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:55.592085 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:55.592108 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:55.592162 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:55.596791 (+ 4629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:55.597753 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:55.597765 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:55.597768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:55.598006 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:55.598466 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:55.598471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:55.598475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:55.598952 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:55.599256 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:55.599588 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 02:57:55.599861 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:57:55.601107 11182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:55.730752 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 11012
I20260808 02:57:55.762156 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 11205
I20260808 02:57:55.780121 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:55.762967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42610 (local address 127.0.205.131:41447)
0808 02:57:55.763170 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:55.763177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:55.763439 (+ 262us) server_negotiation.cc:407] Connection header received
0808 02:57:55.763657 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:55.763662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:55.763801 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:55.763884 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:55.766161 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.766995 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.769154 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:55.769570 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:55.769675 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:55.773614 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:55.773630 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:55.773647 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:55.773692 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:55.776927 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:55.777843 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:55.777852 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:55.777855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:55.778121 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:55.778594 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:55.778597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:55.778599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:55.778971 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:55.779225 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:55.779636 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 02:57:55.779816 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:57:55.781155 11386 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:55.916683 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 11205
I20260808 02:57:55.947468 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 10322
I20260808 02:57:55.948598 10547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:56.019371 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:56.018830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38360)
0808 02:57:56.019036 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:56.019168 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 02:57:56.020437 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:56.020051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34241 (local address 127.0.0.1:38368)
0808 02:57:56.020202 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:56.020287 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 02:57:56.070097 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 10322
I20260808 02:57:56.097774 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 10452
I20260808 02:57:56.099031 10798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:57:56.228977 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 10452
2026-08-08T02:57:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10583 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:57:56 dist-test-slave-6ntk krb5kdc[11579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:57:56 dist-test-slave-6ntk krb5kdc[11579](info): set up 2 sockets
Aug 08 02:57:56 dist-test-slave-6ntk krb5kdc[11579](info): commencing operation
krb5kdc: starting...
W20260808 02:57:57.551203 822 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:57:57 dist-test-slave-6ntk krb5kdc[11579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:57:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:57:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.190.keytab.
Entry for principal oryx/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.190.keytab.
I20260808 02:57:57.826349 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43491
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.190.keytab
--principal=oryx/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:43491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:58.166590 11599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:58.166909 11599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:58.167011 11599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:58.174342 11599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:57:58.174459 11599 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:57:58.174511 11599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:58.174575 11599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:58.174626 11599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:57:58.174691 11599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:57:58.184801 11599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:43491
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.190.keytab
--principal=oryx/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:58.186667 11599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:58.188812 11599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:58.207311 11606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:58.207365 11610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:58.207415 11607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:58.208564 11599 server_base.cc:1034] running on GCE node
I20260808 02:57:58.209568 11599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:58.211222 11599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:58.212500 11599 hybrid_clock.cc:648] HybridClock initialized: now 1786157878212434 us; error 42 us; skew 500 ppm
I20260808 02:57:58.212874 11599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:58 dist-test-slave-6ntk krb5kdc[11579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157878, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:58.216395 11599 init.cc:377] Logged in from keytab as oryx/127.0.205.190@KRBTEST.COM (short username oryx)
I20260808 02:57:58.218156 11599 webserver.cc:460] Webserver started at http://127.0.205.190:33749/ using document root <none> and password file <none>
I20260808 02:57:58.218935 11599 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:58.219069 11599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:58.219445 11599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:58.222769 11599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "6f3402bc91fa4e81af2afd4a46e21bc2"
format_stamp: "Formatted at 2026-08-08 02:57:58 on dist-test-slave-6ntk"
server_key: "036610c391bef74dcb1ab583711e44ce"
server_key_iv: "eefbd3bd21434c563a0cf33f829895b3"
server_key_version: "encryptionkey@0"
I20260808 02:57:58.223661 11599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "6f3402bc91fa4e81af2afd4a46e21bc2"
format_stamp: "Formatted at 2026-08-08 02:57:58 on dist-test-slave-6ntk"
server_key: "036610c391bef74dcb1ab583711e44ce"
server_key_iv: "eefbd3bd21434c563a0cf33f829895b3"
server_key_version: "encryptionkey@0"
I20260808 02:57:58.229312 11599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 02:57:58.233057 11620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:58.234546 11599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:57:58.234733 11599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "6f3402bc91fa4e81af2afd4a46e21bc2"
format_stamp: "Formatted at 2026-08-08 02:57:58 on dist-test-slave-6ntk"
server_key: "036610c391bef74dcb1ab583711e44ce"
server_key_iv: "eefbd3bd21434c563a0cf33f829895b3"
server_key_version: "encryptionkey@0"
I20260808 02:57:58.234980 11599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:58.276525 11599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:58.281280 11599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:58.281581 11599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:58.304423 11599 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:43491
I20260808 02:57:58.304454 11711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:43491 every 8 connection(s)
I20260808 02:57:58.305977 11599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:57:58.309974 11712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:57:58.313043 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 11599
I20260808 02:57:58.313258 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:57:58.313988 822 external_mini_cluster.cc:1428] Setting key 294c3ae9bb94dd67e1309fa95b346ee4
I20260808 02:57:58.323657 11712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2: Bootstrap starting.
I20260808 02:57:58.327411 11712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2: Neither blocks nor log segments found. Creating new log.
I20260808 02:57:58.328476 11712 log.cc:824] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:57:58 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:57:58.331962 11712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2: No bootstrap required, opened a new log
I20260808 02:57:58.339694 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:58.316085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39692 (local address 127.0.205.190:43491)
0808 02:57:58.316922 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:58.316938 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:58.316992 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:57:58.319193 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:58.319229 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:58.319784 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:58.320252 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:58.321598 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:58.323612 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:58.324898 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:58.326078 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:58.326274 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:58.330771 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:58.330782 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:58.330801 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:58.330847 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:58.334965 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:58.335709 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:58.335719 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:58.335734 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:58.335951 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:58.336304 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:58.336309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:58.336312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:58.337076 (+ 764us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:58.337416 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:58.337729 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 02:57:58.338236 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":339,"threads_started":1}
I20260808 02:57:58.342887 11712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3402bc91fa4e81af2afd4a46e21bc2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43491 } }
I20260808 02:57:58.343238 11712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:57:58.343313 11712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f3402bc91fa4e81af2afd4a46e21bc2, State: Initialized, Role: FOLLOWER
I20260808 02:57:58.343811 11712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3402bc91fa4e81af2afd4a46e21bc2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43491 } }
I20260808 02:57:58.344026 11712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:57:58.344121 11712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:57:58.344246 11712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:57:58.346475 11712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3402bc91fa4e81af2afd4a46e21bc2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43491 } }
I20260808 02:57:58.346944 11712 leader_election.cc:302] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f3402bc91fa4e81af2afd4a46e21bc2; no voters:
I20260808 02:57:58.347573 11712 leader_election.cc:288] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:57:58.347702 11720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:57:58.348464 11720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [term 1 LEADER]: Becoming Leader. State: Replica: 6f3402bc91fa4e81af2afd4a46e21bc2, State: Running, Role: LEADER
I20260808 02:57:58.349080 11720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3402bc91fa4e81af2afd4a46e21bc2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43491 } }
I20260808 02:57:58.349794 11712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:57:58.352074 11722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f3402bc91fa4e81af2afd4a46e21bc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3402bc91fa4e81af2afd4a46e21bc2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43491 } } }
I20260808 02:57:58.352455 11722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:58.353102 11723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f3402bc91fa4e81af2afd4a46e21bc2. Latest consensus state: current_term: 1 leader_uuid: "6f3402bc91fa4e81af2afd4a46e21bc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3402bc91fa4e81af2afd4a46e21bc2" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43491 } } }
I20260808 02:57:58.353381 11723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:57:58.358673 11731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:57:58.363946 11731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:57:58.373006 11731 catalog_manager.cc:1347] Generated new cluster ID: 690b86b49d1a446fb89f1a06da41c05a
I20260808 02:57:58.373140 11731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:57:58.385666 11731 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:57:58.387156 11731 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:57:58.396960 11731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2: Generated new TSK 0
I20260808 02:57:58.397921 11731 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab.
Entry for principal oryx/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab.
I20260808 02:57:58.463008 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43491
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab
--principal=oryx/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:58.823637 11751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:58.824023 11751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:58.824223 11751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:58.832123 11751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:58.832304 11751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:58.832466 11751 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:57:58.832614 11751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:57:58.843492 11751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab
--principal=oryx/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.205.190:43491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:58.844944 11751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:58.846475 11751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:58.860812 11757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:58.860937 11759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:58.861131 11761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:58.862052 11751 server_base.cc:1034] running on GCE node
I20260808 02:57:58.862694 11751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:58.863701 11751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:58.864902 11751 hybrid_clock.cc:648] HybridClock initialized: now 1786157878864818 us; error 65 us; skew 500 ppm
I20260808 02:57:58.865173 11751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:58 dist-test-slave-6ntk krb5kdc[11579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157878, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:58.868498 11751 init.cc:377] Logged in from keytab as oryx/127.0.205.129@KRBTEST.COM (short username oryx)
I20260808 02:57:58.870097 11751 webserver.cc:460] Webserver started at http://127.0.205.129:41113/ using document root <none> and password file <none>
I20260808 02:57:58.870867 11751 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:58.870981 11751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:58.871330 11751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:58.874495 11751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "8abc5118a51540278c04093762b87151"
format_stamp: "Formatted at 2026-08-08 02:57:58 on dist-test-slave-6ntk"
server_key: "fc5415b8996cc8e4b6c3fc0f9a9cf2ac"
server_key_iv: "e5a7c83b16bd4e10eec586a2e4b00efd"
server_key_version: "encryptionkey@0"
I20260808 02:57:58.875342 11751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "8abc5118a51540278c04093762b87151"
format_stamp: "Formatted at 2026-08-08 02:57:58 on dist-test-slave-6ntk"
server_key: "fc5415b8996cc8e4b6c3fc0f9a9cf2ac"
server_key_iv: "e5a7c83b16bd4e10eec586a2e4b00efd"
server_key_version: "encryptionkey@0"
I20260808 02:57:58.880894 11751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 02:57:58.884701 11772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:58.886273 11751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:57:58.886471 11751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "8abc5118a51540278c04093762b87151"
format_stamp: "Formatted at 2026-08-08 02:57:58 on dist-test-slave-6ntk"
server_key: "fc5415b8996cc8e4b6c3fc0f9a9cf2ac"
server_key_iv: "e5a7c83b16bd4e10eec586a2e4b00efd"
server_key_version: "encryptionkey@0"
I20260808 02:57:58.886714 11751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:58.903937 11751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:58.909085 11751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:58.909391 11751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:58.912283 11751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:58.912425 11751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:58.912585 11751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:58.912665 11751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:57:58 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157878, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:57:58.942345 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:58.913243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:37739 (local address 127.0.205.190:43491)
0808 02:57:58.913772 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:58.913779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:58.915106 (+ 1327us) server_negotiation.cc:407] Connection header received
0808 02:57:58.918656 (+ 3550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:58.918665 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:58.918782 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:58.918938 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:58.922084 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:58.923100 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:58.926616 (+ 3516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:58.927061 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:58.927224 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:58.933463 (+ 6239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:58.933470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:58.933473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:58.933514 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:58.937372 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:58.938775 (+ 1403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:58.938780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:58.938783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:58.938950 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:58.939598 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:58.939604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:58.939607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:58.940052 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:58.940221 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:58.941325 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0808 02:57:58.941689 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":267,"threads_started":1}
I20260808 02:57:58.943190 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:58.913971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43491 (local address 127.0.205.129:37739)
0808 02:57:58.914888 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:58.914947 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:58.918206 (+ 3259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:58.919314 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:58.919344 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:58.920503 (+ 1159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:58.921770 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:58.921812 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:58.923346 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:58.923357 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:58.926327 (+ 2970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:58.926438 (+ 111us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:58.927370 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:58.927379 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:58.927621 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:58.928469 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:58.928517 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:58.933139 (+ 4622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:58.937589 (+ 4450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:58.937602 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:58.937630 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:58.938601 (+ 971us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:58.939131 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:58.939139 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:58.939143 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:58.939460 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:58.940297 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:58.940303 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:58.940753 (+ 450us) client_negotiation.cc:769] Sending connection context
0808 02:57:58.941088 (+ 335us) client_negotiation.cc:240] Negotiation successful
0808 02:57:58.941805 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":330,"threads_started":1}
I20260808 02:57:58.965534 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:58.949584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43491 (local address 127.0.205.129:49747)
0808 02:57:58.950310 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:58.950329 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:58.950488 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:58.950877 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:58.950881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:58.951424 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:58.951729 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:58.951738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:58.952830 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:58.952835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:58.954442 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:58.954480 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:58.955656 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:58.955661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:58.955906 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:58.956670 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:58.956688 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:58.957449 (+ 761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:58.962580 (+ 5131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:58.962584 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:58.962588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:58.963025 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:58.963537 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:58.963540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:58.963543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:58.963664 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:58.964437 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:58.964441 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:58.964618 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 02:57:58.964843 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 02:57:58.965185 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":426,"threads_started":1}
I20260808 02:57:58.965943 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:58.949463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:49747 (local address 127.0.205.190:43491)
0808 02:57:58.949633 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:58.949638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:58.950413 (+ 775us) server_negotiation.cc:407] Connection header received
0808 02:57:58.950670 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:58.950674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:58.950737 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:58.950840 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:58.951906 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:58.952691 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:58.954669 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:58.955226 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:58.955345 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:58.957670 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:58.957679 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:58.957686 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:58.957733 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:58.962414 (+ 4681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:58.963174 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:58.963184 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:58.963191 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:58.963415 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:58.963778 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:58.963786 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:58.963793 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:58.964309 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:58.964501 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:58.964982 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 02:57:58.965357 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:57:58.980212 11751 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:43055
I20260808 02:57:58.980221 11946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:43055 every 8 connection(s)
I20260808 02:57:58.981573 11751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:57:58.983201 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 11751
I20260808 02:57:58.983424 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:57:58.984208 822 external_mini_cluster.cc:1428] Setting key d67e3f92b346e2ce9ce9d625b0b6d886
I20260808 02:57:58.983868 11947 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43491
I20260808 02:57:58.985549 11947 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:58.986510 11947 heartbeater.cc:511] Master 127.0.205.190:43491 requested a full tablet report, sending...
I20260808 02:57:58.989111 11637 ts_manager.cc:194] Registered new tserver with Master: 8abc5118a51540278c04093762b87151 (127.0.205.129:43055)
I20260808 02:57:58.991649 11637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:37739
WARNING: no policy specified for oryx/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab.
Entry for principal oryx/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab.
I20260808 02:57:59.045338 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43491
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab
--principal=oryx/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:59.406342 11952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:59.406653 11952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:59.406812 11952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:59.414446 11952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:59.414582 11952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:59.414752 11952 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:57:59.414877 11952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:57:59.425107 11952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab
--principal=oryx/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.205.190:43491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:59.426597 11952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:59.428022 11952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:57:59.442328 11960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:59.442356 11962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:57:59.443082 11959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:57:59.443164 11952 server_base.cc:1034] running on GCE node
I20260808 02:57:59.444016 11952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:57:59.445047 11952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:57:59.446287 11952 hybrid_clock.cc:648] HybridClock initialized: now 1786157879446217 us; error 44 us; skew 500 ppm
I20260808 02:57:59.446633 11952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:57:59 dist-test-slave-6ntk krb5kdc[11579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157879, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:57:59.450055 11952 init.cc:377] Logged in from keytab as oryx/127.0.205.130@KRBTEST.COM (short username oryx)
I20260808 02:57:59.451678 11952 webserver.cc:460] Webserver started at http://127.0.205.130:43913/ using document root <none> and password file <none>
I20260808 02:57:59.452425 11952 fs_manager.cc:359] Metadata directory not provided
I20260808 02:57:59.452544 11952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:57:59.452898 11952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:57:59.456007 11952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "5af025c42d7a4c36ac51c6495b535398"
format_stamp: "Formatted at 2026-08-08 02:57:59 on dist-test-slave-6ntk"
server_key: "8cdc93c05988bd402db5aeff4ec92621"
server_key_iv: "c8df063043a8fa4cb741ebcc299d2930"
server_key_version: "encryptionkey@0"
I20260808 02:57:59.456838 11952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "5af025c42d7a4c36ac51c6495b535398"
format_stamp: "Formatted at 2026-08-08 02:57:59 on dist-test-slave-6ntk"
server_key: "8cdc93c05988bd402db5aeff4ec92621"
server_key_iv: "c8df063043a8fa4cb741ebcc299d2930"
server_key_version: "encryptionkey@0"
I20260808 02:57:59.462824 11952 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260808 02:57:59.466595 11971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:59.467978 11952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:57:59.468164 11952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "5af025c42d7a4c36ac51c6495b535398"
format_stamp: "Formatted at 2026-08-08 02:57:59 on dist-test-slave-6ntk"
server_key: "8cdc93c05988bd402db5aeff4ec92621"
server_key_iv: "c8df063043a8fa4cb741ebcc299d2930"
server_key_version: "encryptionkey@0"
I20260808 02:57:59.468411 11952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:57:59.487250 11952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:57:59.492012 11952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:57:59.492295 11952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:57:59.494956 11952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:57:59.495075 11952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:57:59.495223 11952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:57:59.495293 11952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:57:59 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157879, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:57:59.526057 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:59.496693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43491 (local address 127.0.205.130:36801)
0808 02:57:59.497476 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:59.497520 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:59.499748 (+ 2228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:59.500939 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:59.500975 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:59.502038 (+ 1063us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:59.503106 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:59.503140 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:59.504717 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:59.504726 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:59.508244 (+ 3518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:59.508276 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:59.509409 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:59.509420 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:59.509890 (+ 470us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:59.510970 (+ 1080us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:59.511022 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:59.515922 (+ 4900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:59.520296 (+ 4374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:59.520306 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:59.520329 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:59.521076 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:59.521577 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:59.521589 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:59.521594 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:59.521922 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:59.522621 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:59.522633 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:59.523232 (+ 599us) client_negotiation.cc:769] Sending connection context
0808 02:57:59.523579 (+ 347us) client_negotiation.cc:240] Negotiation successful
0808 02:57:59.524231 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":316,"threads_started":1}
I20260808 02:57:59.526780 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:59.495905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:36801 (local address 127.0.205.190:43491)
0808 02:57:59.496463 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:59.496468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:59.497634 (+ 1166us) server_negotiation.cc:407] Connection header received
0808 02:57:59.500144 (+ 2510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:59.500149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:59.500243 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:59.500345 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:59.503318 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:59.504525 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:59.508483 (+ 3958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:59.509059 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:59.509235 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:59.516339 (+ 7104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:59.516349 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:59.516356 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:59.516420 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:59.520125 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:59.521231 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:59.521237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:59.521241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:59.521408 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:59.522057 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:59.522062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:59.522066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:59.522472 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:59.522656 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:59.526118 (+ 3462us) server_negotiation.cc:299] Negotiation successful
0808 02:57:59.526387 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":311,"threads_started":1}
I20260808 02:57:59.541236 12138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:46157 every 8 connection(s)
I20260808 02:57:59.541234 11952 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:46157
I20260808 02:57:59.543020 11952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:57:59.545504 12143 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43491
I20260808 02:57:59.545769 12143 heartbeater.cc:464] Registering TS with master...
I20260808 02:57:59.546788 12143 heartbeater.cc:511] Master 127.0.205.190:43491 requested a full tablet report, sending...
I20260808 02:57:59.548637 11637 ts_manager.cc:194] Registered new tserver with Master: 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157)
I20260808 02:57:59.549008 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:59.533833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:56687 (local address 127.0.205.190:43491)
0808 02:57:59.534086 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 02:57:59.534094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:57:59.534873 (+ 779us) server_negotiation.cc:407] Connection header received
0808 02:57:59.535215 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:57:59.535225 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:57:59.535355 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:57:59.535510 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:57:59.536460 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:59.537198 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:59.538924 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:57:59.539753 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:57:59.539943 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:59.542553 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:57:59.542558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:57:59.542561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:57:59.542595 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:57:59.546058 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:59.547003 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:59.547007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:59.547009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:59.547129 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:57:59.547543 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:57:59.547546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:57:59.547548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:57:59.547977 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:57:59.548092 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:57:59.548590 (+ 498us) server_negotiation.cc:299] Negotiation successful
0808 02:57:59.548769 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260808 02:57:59.549329 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:57:59.533923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43491 (local address 127.0.205.130:56687)
0808 02:57:59.534745 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0808 02:57:59.534762 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:57:59.535011 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:57:59.535517 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:57:59.535521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:57:59.535959 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:57:59.536306 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:59.536318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:59.537358 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:59.537363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:59.538743 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:57:59.538755 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:57:59.540011 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:57:59.540024 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:57:59.540211 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:57:59.540963 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:57:59.541005 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:57:59.542317 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:57:59.546186 (+ 3869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:59.546191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:59.546195 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:59.546873 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:59.547245 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:57:59.547253 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:57:59.547258 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:57:59.547425 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:57:59.548103 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:57:59.548108 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:57:59.548335 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 02:57:59.548500 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 02:57:59.548871 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":459,"threads_started":1}
I20260808 02:57:59.549852 11637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:36801
I20260808 02:57:59.552474 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 11952
I20260808 02:57:59.552670 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:57:59.553239 822 external_mini_cluster.cc:1428] Setting key a6f6b9ea73a2976a079f84d564e30c0b
WARNING: no policy specified for oryx/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab.
Entry for principal oryx/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab.
I20260808 02:57:59.613818 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43491
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab
--principal=oryx/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:57:59.965243 12148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:57:59.965581 12148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:57:59.965823 12148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:57:59.973533 12148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:57:59.973681 12148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:57:59.973816 12148 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:57:59.973969 12148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:57:59.984153 12148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab
--principal=oryx/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.205.190:43491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:57:59.985409 12148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:57:59.986780 12148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:57:59.995635 11947 heartbeater.cc:503] Master 127.0.205.190:43491 was elected leader, sending a full tablet report...
W20260808 02:58:00.000643 12155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:00.000805 12159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:00.000836 12157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:00.001652 12148 server_base.cc:1034] running on GCE node
I20260808 02:58:00.002370 12148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:00.003343 12148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:00.004603 12148 hybrid_clock.cc:648] HybridClock initialized: now 1786157880004523 us; error 55 us; skew 500 ppm
I20260808 02:58:00.004930 12148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:00 dist-test-slave-6ntk krb5kdc[11579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157880, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:00.008492 12148 init.cc:377] Logged in from keytab as oryx/127.0.205.131@KRBTEST.COM (short username oryx)
I20260808 02:58:00.010133 12148 webserver.cc:460] Webserver started at http://127.0.205.131:43127/ using document root <none> and password file <none>
I20260808 02:58:00.010910 12148 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:00.011049 12148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:00.011395 12148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:00.014495 12148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "1aabdcbc57d64706bf48c93e177d18f3"
format_stamp: "Formatted at 2026-08-08 02:58:00 on dist-test-slave-6ntk"
server_key: "97c0e3146041ce955c184b67aa7b3f68"
server_key_iv: "64a5f3cddd747ede1c80a3f792aeb88d"
server_key_version: "encryptionkey@0"
I20260808 02:58:00.015404 12148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "1aabdcbc57d64706bf48c93e177d18f3"
format_stamp: "Formatted at 2026-08-08 02:58:00 on dist-test-slave-6ntk"
server_key: "97c0e3146041ce955c184b67aa7b3f68"
server_key_iv: "64a5f3cddd747ede1c80a3f792aeb88d"
server_key_version: "encryptionkey@0"
I20260808 02:58:00.020867 12148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:58:00.024446 12168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:00.025751 12148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:58:00.025979 12148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "1aabdcbc57d64706bf48c93e177d18f3"
format_stamp: "Formatted at 2026-08-08 02:58:00 on dist-test-slave-6ntk"
server_key: "97c0e3146041ce955c184b67aa7b3f68"
server_key_iv: "64a5f3cddd747ede1c80a3f792aeb88d"
server_key_version: "encryptionkey@0"
I20260808 02:58:00.026209 12148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:00.059814 12148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:00.064482 12148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:00.064777 12148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:00.067677 12148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:00.067785 12148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:00.067930 12148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:00.068020 12148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:58:00 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157880, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:58:00.095100 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:00.068288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:46157 (local address 127.0.205.190:43491)
0808 02:58:00.069006 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:00.069011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:00.070093 (+ 1082us) server_negotiation.cc:407] Connection header received
0808 02:58:00.072543 (+ 2450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:00.072549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:00.072649 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:00.072789 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:00.075789 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:00.076805 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:00.080266 (+ 3461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:00.080853 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:00.080997 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:00.087025 (+ 6028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:00.087031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:00.087034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:00.087071 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:00.090790 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:00.092176 (+ 1386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:00.092182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:00.092186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:00.092350 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:00.092998 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:00.093002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:00.093004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:00.093281 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:00.093413 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:00.094614 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0808 02:58:00.094797 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":499,"threads_started":1}
I20260808 02:58:00.097275 12183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:00.069005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43491 (local address 127.0.205.131:46157)
0808 02:58:00.069887 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:00.069948 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:00.072185 (+ 2237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:00.073267 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:00.073282 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:00.074382 (+ 1100us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:00.075554 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:00.075587 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:00.077045 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:00.077052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:00.080053 (+ 3001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:00.080085 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:00.081339 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:00.081350 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:00.081681 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:00.082618 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:00.082641 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:00.086702 (+ 4061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:00.090967 (+ 4265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:00.090981 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:00.091004 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:00.092001 (+ 997us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:00.092484 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:00.092492 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:00.092498 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:00.092857 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:00.093422 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:00.093434 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:00.094093 (+ 659us) client_negotiation.cc:769] Sending connection context
0808 02:58:00.094511 (+ 418us) client_negotiation.cc:240] Negotiation successful
0808 02:58:00.095276 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":349,"threads_started":1}
I20260808 02:58:00.119856 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:00.103437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:49355 (local address 127.0.205.190:43491)
0808 02:58:00.103630 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:00.103638 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:00.104531 (+ 893us) server_negotiation.cc:407] Connection header received
0808 02:58:00.104811 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:00.104817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:00.104918 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:00.105036 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:00.106174 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:00.107462 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:00.109756 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:00.110469 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:00.110626 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:00.112642 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:00.112652 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:00.112659 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:00.112714 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:00.116424 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:00.117417 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:00.117426 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:00.117430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:00.117581 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:00.118139 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:00.118143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:00.118145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:00.118546 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:00.118715 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:00.119359 (+ 644us) server_negotiation.cc:299] Negotiation successful
0808 02:58:00.119576 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:58:00.120491 12285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:00.103587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43491 (local address 127.0.205.131:49355)
0808 02:58:00.104380 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:00.104408 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:00.104620 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:00.105064 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:00.105069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:00.105612 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:00.106014 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:00.106026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:00.107678 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:00.107684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:00.109576 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:00.109600 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:00.110698 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:00.110706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:00.110899 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:00.111589 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:00.111606 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:00.112400 (+ 794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:00.116568 (+ 4168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:00.116574 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:00.116581 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:00.117274 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:00.117699 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:00.117705 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:00.117711 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:00.117960 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:00.118728 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:00.118747 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:00.119031 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 02:58:00.119261 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 02:58:00.119798 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":411,"threads_started":1}
I20260808 02:58:00.123713 12148 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:35453
I20260808 02:58:00.123719 12337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:35453 every 8 connection(s)
I20260808 02:58:00.125383 12148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:58:00.127538 12338 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43491
I20260808 02:58:00.127779 12338 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:00.128451 12338 heartbeater.cc:511] Master 127.0.205.190:43491 requested a full tablet report, sending...
I20260808 02:58:00.129846 11636 ts_manager.cc:194] Registered new tserver with Master: 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453)
I20260808 02:58:00.130885 11636 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:46157
I20260808 02:58:00.131433 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 12148
I20260808 02:58:00.131618 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:58:00.132129 822 external_mini_cluster.cc:1428] Setting key bdeac93e4a6be4bf7632614d80511542
I20260808 02:58:00.135071 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:58:00 dist-test-slave-6ntk krb5kdc[11579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157880, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:58:00 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:00 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:00.552860 12143 heartbeater.cc:503] Master 127.0.205.190:43491 was elected leader, sending a full tablet report...
I20260808 02:58:01.133479 12338 heartbeater.cc:503] Master 127.0.205.190:43491 was elected leader, sending a full tablet report...
W20260808 02:58:01.406535 12348 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.212s user 0.000s sys 0.021s
W20260808 02:58:01.407955 12182 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:00.186167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39708 (local address 127.0.205.190:43491)
0808 02:58:00.186438 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:00.186446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:00.186859 (+ 413us) server_negotiation.cc:407] Connection header received
0808 02:58:00.187179 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:00.187188 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:00.187295 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:00.187428 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:00.189607 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:00.190736 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:00.192695 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:00.193120 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:00.193267 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.407578 (+1214311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39708: BlockingRecv error: recv got EOF from 127.0.0.1:39708 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 08 02:58:01 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157880, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:58:01.430522 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.412624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52572 (local address 127.0.205.190:43491)
0808 02:58:01.412867 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:01.412876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:01.413033 (+ 157us) server_negotiation.cc:407] Connection header received
0808 02:58:01.413192 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:01.413196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:01.413265 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:01.413379 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:01.414738 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.415459 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.416722 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.417159 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.417263 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.424813 (+ 7550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:01.424820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:01.424823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:01.424862 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:01.427779 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:01.428545 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:01.428550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:01.428552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:01.428707 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:01.429120 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:01.429124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:01.429126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:01.429433 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:01.429647 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:01.429998 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 02:58:01.430243 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 02:58:01.435079 11636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:58:01.442361 11636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:58:01.476022 12370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.455572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:35453 (local address 127.0.0.1:43288)
0808 02:58:01.466422 (+ 10850us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:01.466449 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:01.466612 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:01.468053 (+ 1441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:01.468064 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:01.468100 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:01.469154 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.469166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.470894 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.470899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.473107 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.473123 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.474849 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.474855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.475049 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.475118 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 02:58:01.475305 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 02:58:01.475511 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10680,"mutex_wait_us":77,"thread_start_us":12154,"threads_started":1}
I20260808 02:58:01.476430 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.455687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46162 (local address 127.0.205.129:43055)
0808 02:58:01.467545 (+ 11858us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:01.467553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:01.467579 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:58:01.467672 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:01.467680 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:01.467886 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:01.468460 (+ 574us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:01.468871 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.470442 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.473616 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.475114 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.475297 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.475475 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:01.475737 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 02:58:01.475914 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11708,"thread_start_us":11564,"threads_started":1}
I20260808 02:58:01.476552 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.458023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.0.205.130:46157)
0808 02:58:01.458603 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:01.458615 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:01.468137 (+ 9522us) server_negotiation.cc:407] Connection header received
0808 02:58:01.468469 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:01.468477 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:01.468721 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:01.468887 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:01.469661 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.470773 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.473691 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.475114 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.475273 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.475475 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:01.476107 (+ 632us) server_negotiation.cc:299] Negotiation successful
0808 02:58:01.476275 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":257,"threads_started":1}
I20260808 02:58:01.476743 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.456052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.0.205.131:35453)
0808 02:58:01.467194 (+ 11142us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:01.467204 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:01.467239 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:58:01.467358 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:01.467366 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:01.467669 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:01.467880 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:01.469350 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.470747 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.473364 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.474637 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.474792 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.475030 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:01.476297 (+ 1267us) server_negotiation.cc:299] Negotiation successful
0808 02:58:01.476430 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10964,"thread_start_us":10803,"threads_started":1}
I20260808 02:58:01.476188 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.455501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43055 (local address 127.0.0.1:46162)
0808 02:58:01.456407 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:01.456448 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:01.456615 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:01.468115 (+ 11500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:01.468120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:01.468146 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:01.468689 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.468702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.470587 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.470592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.473405 (+ 2813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.473421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.475358 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.475364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.475538 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.475567 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:58:01.475678 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 02:58:01.475804 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":191,"thread_start_us":750,"threads_started":1}
I20260808 02:58:01.477584 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.457670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:46157 (local address 127.0.0.1:34682)
0808 02:58:01.468004 (+ 10334us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:01.468022 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:01.468254 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:01.469167 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:01.469171 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:01.469200 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:01.469497 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.469504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.470979 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.470989 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.473525 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.473536 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.475699 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.475707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.475891 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.475926 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:58:01.476784 (+ 858us) client_negotiation.cc:240] Negotiation successful
0808 02:58:01.476923 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10200,"mutex_wait_us":8,"thread_start_us":8435,"threads_started":1}
I20260808 02:58:01.480626 11841 tablet_service.cc:1467] Processing CreateTablet for tablet a4f7c7d934654e40a8f8a439eac3348a (DEFAULT_TABLE table=test-table [id=d70684f7888147aeae87627b2bdbf098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:01.480726 12238 tablet_service.cc:1467] Processing CreateTablet for tablet a4f7c7d934654e40a8f8a439eac3348a (DEFAULT_TABLE table=test-table [id=d70684f7888147aeae87627b2bdbf098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:01.480962 12042 tablet_service.cc:1467] Processing CreateTablet for tablet a4f7c7d934654e40a8f8a439eac3348a (DEFAULT_TABLE table=test-table [id=d70684f7888147aeae87627b2bdbf098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:01.482241 12042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f7c7d934654e40a8f8a439eac3348a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:01.482240 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f7c7d934654e40a8f8a439eac3348a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:01.482816 11841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f7c7d934654e40a8f8a439eac3348a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:01.500725 12373 tablet_bootstrap.cc:492] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: Bootstrap starting.
I20260808 02:58:01.502959 12376 tablet_bootstrap.cc:492] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: Bootstrap starting.
I20260808 02:58:01.504659 12373 tablet_bootstrap.cc:654] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:01.504762 12372 tablet_bootstrap.cc:492] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: Bootstrap starting.
I20260808 02:58:01.506125 12373 log.cc:824] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: Log is configured to *not* fsync() on all Append() calls
I20260808 02:58:01.506860 12376 tablet_bootstrap.cc:654] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:01.508351 12372 tablet_bootstrap.cc:654] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:01.508409 12376 log.cc:824] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:58:01.509763 12372 log.cc:824] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: Log is configured to *not* fsync() on all Append() calls
I20260808 02:58:01.513360 12373 tablet_bootstrap.cc:492] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: No bootstrap required, opened a new log
I20260808 02:58:01.513373 12376 tablet_bootstrap.cc:492] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: No bootstrap required, opened a new log
I20260808 02:58:01.513475 12372 tablet_bootstrap.cc:492] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: No bootstrap required, opened a new log
I20260808 02:58:01.513682 12376 ts_tablet_manager.cc:1397] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260808 02:58:01.513715 12373 ts_tablet_manager.cc:1397] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260808 02:58:01.513764 12372 ts_tablet_manager.cc:1397] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260808 02:58:01.525046 12372 raft_consensus.cc:348] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } }
I20260808 02:58:01.525804 12372 raft_consensus.cc:374] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:01.526028 12372 raft_consensus.cc:729] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af025c42d7a4c36ac51c6495b535398, State: Initialized, Role: FOLLOWER
I20260808 02:58:01.526682 12373 raft_consensus.cc:348] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } }
I20260808 02:58:01.526937 12372 consensus_queue.cc:260] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } }
I20260808 02:58:01.527205 12373 raft_consensus.cc:374] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:01.527305 12373 raft_consensus.cc:729] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8abc5118a51540278c04093762b87151, State: Initialized, Role: FOLLOWER
I20260808 02:58:01.527964 12373 consensus_queue.cc:260] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } }
I20260808 02:58:01.529451 12372 ts_tablet_manager.cc:1428] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260808 02:58:01.529738 12373 ts_tablet_manager.cc:1428] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: Time spent starting tablet: real 0.016s user 0.010s sys 0.003s
I20260808 02:58:01.527873 12376 raft_consensus.cc:348] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } }
I20260808 02:58:01.532276 12376 raft_consensus.cc:374] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:01.532469 12376 raft_consensus.cc:729] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aabdcbc57d64706bf48c93e177d18f3, State: Initialized, Role: FOLLOWER
I20260808 02:58:01.533252 12376 consensus_queue.cc:260] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } }
I20260808 02:58:01.538306 12376 ts_tablet_manager.cc:1428] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: Time spent starting tablet: real 0.024s user 0.016s sys 0.002s
W20260808 02:58:01.546161 12145 tablet.cc:2369] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:58:01.628288 12339 tablet.cc:2369] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:58:01.680373 12383 raft_consensus.cc:482] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:58:01.680676 12383 raft_consensus.cc:504] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } }
I20260808 02:58:01.682646 12383 leader_election.cc:288] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8abc5118a51540278c04093762b87151 (127.0.205.129:43055), 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157)
I20260808 02:58:01.689553 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.683153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:45537 (local address 127.0.205.129:43055)
0808 02:58:01.683420 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:01.683430 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:01.684232 (+ 802us) server_negotiation.cc:407] Connection header received
0808 02:58:01.684463 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:01.684467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:01.684527 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:01.684624 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:01.685230 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.686017 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.687819 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.688615 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.688816 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.688910 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:01.689159 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 02:58:01.689296 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 02:58:01.689669 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.683162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:34895 (local address 127.0.205.130:46157)
0808 02:58:01.683405 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:01.683413 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:01.684288 (+ 875us) server_negotiation.cc:407] Connection header received
0808 02:58:01.684509 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:01.684513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:01.684575 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:01.684674 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:01.685389 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.686186 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.687974 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.688739 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.688933 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.689013 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:01.689280 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 02:58:01.689422 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:58:01.689764 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.683158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:46157 (local address 127.0.205.131:34895)
0808 02:58:01.684166 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:01.684185 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:01.684339 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:01.684695 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:01.684699 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:01.684724 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:01.685220 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.685229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.686326 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.686331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.687811 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.687823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.688968 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.688975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.689121 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.689143 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 02:58:01.689243 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 02:58:01.689376 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"thread_start_us":617,"threads_started":1}
I20260808 02:58:01.689786 12384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.683112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43055 (local address 127.0.205.131:45537)
0808 02:58:01.684106 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:01.684123 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:01.684303 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:01.684641 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:01.684645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:01.684686 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:01.685075 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.685083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.686141 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.686149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.687654 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.687665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.688824 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.688830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.689003 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.689026 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 02:58:01.689133 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 02:58:01.689262 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":5,"thread_start_us":559,"threads_started":1}
I20260808 02:58:01.691002 12071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f7c7d934654e40a8f8a439eac3348a" candidate_uuid: "1aabdcbc57d64706bf48c93e177d18f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af025c42d7a4c36ac51c6495b535398" is_pre_election: true
I20260808 02:58:01.691136 11880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f7c7d934654e40a8f8a439eac3348a" candidate_uuid: "1aabdcbc57d64706bf48c93e177d18f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8abc5118a51540278c04093762b87151" is_pre_election: true
I20260808 02:58:01.691578 12071 raft_consensus.cc:2454] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aabdcbc57d64706bf48c93e177d18f3 in term 0.
I20260808 02:58:01.691676 11880 raft_consensus.cc:2454] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aabdcbc57d64706bf48c93e177d18f3 in term 0.
I20260808 02:58:01.692312 12170 leader_election.cc:302] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aabdcbc57d64706bf48c93e177d18f3, 5af025c42d7a4c36ac51c6495b535398; no voters:
I20260808 02:58:01.692705 12383 raft_consensus.cc:2781] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:58:01.692873 12383 raft_consensus.cc:482] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:58:01.693006 12383 raft_consensus.cc:3037] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:01.695576 12383 raft_consensus.cc:504] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } }
I20260808 02:58:01.696233 12383 leader_election.cc:288] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [CANDIDATE]: Term 1 election: Requested vote from peers 8abc5118a51540278c04093762b87151 (127.0.205.129:43055), 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157)
I20260808 02:58:01.696624 11880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f7c7d934654e40a8f8a439eac3348a" candidate_uuid: "1aabdcbc57d64706bf48c93e177d18f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8abc5118a51540278c04093762b87151"
I20260808 02:58:01.696729 12071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f7c7d934654e40a8f8a439eac3348a" candidate_uuid: "1aabdcbc57d64706bf48c93e177d18f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af025c42d7a4c36ac51c6495b535398"
I20260808 02:58:01.696841 11880 raft_consensus.cc:3037] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:01.696980 12071 raft_consensus.cc:3037] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:01.699198 11880 raft_consensus.cc:2454] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aabdcbc57d64706bf48c93e177d18f3 in term 1.
I20260808 02:58:01.699294 12071 raft_consensus.cc:2454] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aabdcbc57d64706bf48c93e177d18f3 in term 1.
I20260808 02:58:01.699645 12173 leader_election.cc:302] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aabdcbc57d64706bf48c93e177d18f3, 8abc5118a51540278c04093762b87151; no voters:
I20260808 02:58:01.699955 12383 raft_consensus.cc:2781] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:58:01.700439 12383 raft_consensus.cc:686] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 LEADER]: Becoming Leader. State: Replica: 1aabdcbc57d64706bf48c93e177d18f3, State: Running, Role: LEADER
I20260808 02:58:01.700966 12383 consensus_queue.cc:237] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } }
I20260808 02:58:01.706102 11637 catalog_manager.cc:5495] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131). New cstate: current_term: 1 leader_uuid: "1aabdcbc57d64706bf48c93e177d18f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } health_report { overall_health: UNKNOWN } } }
W20260808 02:58:01.735471 11948 tablet.cc:2369] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:58:01.764220 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.758562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.0.205.131:35453)
0808 02:58:01.758804 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:01.758812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:01.758925 (+ 113us) server_negotiation.cc:407] Connection header received
0808 02:58:01.759186 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:01.759190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:01.759277 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:01.759450 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:58:01.759870 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.760571 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.761996 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.762434 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.762572 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.763022 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:58:01.763589 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:58:01.763756 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:01.763838 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 02:58:01.763926 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 02:58:01.772878 11880 raft_consensus.cc:1261] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Refusing update from remote peer 1aabdcbc57d64706bf48c93e177d18f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:01.772903 12071 raft_consensus.cc:1261] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Refusing update from remote peer 1aabdcbc57d64706bf48c93e177d18f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:01.773984 12386 consensus_queue.cc:1035] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:01.774358 12383 consensus_queue.cc:1035] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:01.786526 12391 mvcc.cc:205] Tried to move back new op lower bound from 7316102683731202048 to 7316102683451256832. Current Snapshot: MvccSnapshot[applied={T|T < 7316102683731202048}]
I20260808 02:58:01.788676 12394 mvcc.cc:205] Tried to move back new op lower bound from 7316102683731202048 to 7316102683451256832. Current Snapshot: MvccSnapshot[applied={T|T < 7316102683731202048}]
I20260808 02:58:01.802124 11637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:58:01.802412 11637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:58:01.806066 11637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2: Sending DeleteTablet for 3 replicas of tablet a4f7c7d934654e40a8f8a439eac3348a
I20260808 02:58:01.807230 11841 tablet_service.cc:1514] Processing DeleteTablet for tablet a4f7c7d934654e40a8f8a439eac3348a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:01 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:46162
I20260808 02:58:01.807731 12238 tablet_service.cc:1514] Processing DeleteTablet for tablet a4f7c7d934654e40a8f8a439eac3348a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:01 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:43288
I20260808 02:58:01.808765 12042 tablet_service.cc:1514] Processing DeleteTablet for tablet a4f7c7d934654e40a8f8a439eac3348a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:01 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:34682
I20260808 02:58:01.809098 12412 tablet_replica.cc:330] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: stopping tablet replica
I20260808 02:58:01.809546 12412 raft_consensus.cc:2229] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:58:01.810453 12412 raft_consensus.cc:2258] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:01.811782 11637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 02:58:01.812656 12412 ts_tablet_manager.cc:1905] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:58:01.813830 11637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:58:01.820508 12412 ts_tablet_manager.cc:1918] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:58:01.820756 12412 log.cc:1196] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/a4f7c7d934654e40a8f8a439eac3348a
I20260808 02:58:01.821364 12412 ts_tablet_manager.cc:1939] T a4f7c7d934654e40a8f8a439eac3348a P 1aabdcbc57d64706bf48c93e177d18f3: Deleting consensus metadata
I20260808 02:58:01.823482 11622 catalog_manager.cc:4841] TS 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453): tablet a4f7c7d934654e40a8f8a439eac3348a (table test-table [id=d70684f7888147aeae87627b2bdbf098]) successfully deleted
I20260808 02:58:01.824311 12413 tablet_replica.cc:330] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: stopping tablet replica
I20260808 02:58:01.824806 11838 tablet_service.cc:1467] Processing CreateTablet for tablet b2a3ead846dd435f8fdec9a7add584ca (DEFAULT_TABLE table=test-table [id=e1fd4274a862414ba04f23e83d112228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:01.825387 11838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2a3ead846dd435f8fdec9a7add584ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:01.825347 12238 tablet_service.cc:1467] Processing CreateTablet for tablet b2a3ead846dd435f8fdec9a7add584ca (DEFAULT_TABLE table=test-table [id=e1fd4274a862414ba04f23e83d112228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:01.825946 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2a3ead846dd435f8fdec9a7add584ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:01.829181 12411 tablet_replica.cc:330] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: stopping tablet replica
I20260808 02:58:01.829701 12411 raft_consensus.cc:2229] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:58:01.830421 12411 raft_consensus.cc:2258] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:01.832836 12411 ts_tablet_manager.cc:1905] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:01.832942 12373 tablet_bootstrap.cc:492] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151: Bootstrap starting.
I20260808 02:58:01.835212 12373 tablet_bootstrap.cc:654] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:01.837984 12376 tablet_bootstrap.cc:492] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3: Bootstrap starting.
I20260808 02:58:01.824988 12042 tablet_service.cc:1467] Processing CreateTablet for tablet b2a3ead846dd435f8fdec9a7add584ca (DEFAULT_TABLE table=test-table [id=e1fd4274a862414ba04f23e83d112228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:01.838461 12042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2a3ead846dd435f8fdec9a7add584ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:01.839557 12376 tablet_bootstrap.cc:654] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:01.841450 12411 ts_tablet_manager.cc:1918] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:58:01.842113 12411 log.cc:1196] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/a4f7c7d934654e40a8f8a439eac3348a
I20260808 02:58:01.842329 12413 raft_consensus.cc:2229] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:58:01.842921 12411 ts_tablet_manager.cc:1939] T a4f7c7d934654e40a8f8a439eac3348a P 8abc5118a51540278c04093762b87151: Deleting consensus metadata
I20260808 02:58:01.844782 12413 raft_consensus.cc:2258] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:01.848133 12372 tablet_bootstrap.cc:492] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398: Bootstrap starting.
I20260808 02:58:01.849108 11623 catalog_manager.cc:4841] TS 8abc5118a51540278c04093762b87151 (127.0.205.129:43055): tablet a4f7c7d934654e40a8f8a439eac3348a (table test-table [id=d70684f7888147aeae87627b2bdbf098]) successfully deleted
I20260808 02:58:01.850248 12372 tablet_bootstrap.cc:654] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:01.852017 12373 tablet_bootstrap.cc:492] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151: No bootstrap required, opened a new log
I20260808 02:58:01.852238 12373 ts_tablet_manager.cc:1397] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.004s
I20260808 02:58:01.852855 12376 tablet_bootstrap.cc:492] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3: No bootstrap required, opened a new log
I20260808 02:58:01.853060 12376 ts_tablet_manager.cc:1397] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.008s
I20260808 02:58:01.853405 12373 raft_consensus.cc:348] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:01.853696 12373 raft_consensus.cc:374] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:01.853827 12373 raft_consensus.cc:729] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8abc5118a51540278c04093762b87151, State: Initialized, Role: FOLLOWER
I20260808 02:58:01.854202 12413 ts_tablet_manager.cc:1905] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:01.854434 12373 consensus_queue.cc:260] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:01.855360 12372 tablet_bootstrap.cc:492] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398: No bootstrap required, opened a new log
I20260808 02:58:01.855607 12372 ts_tablet_manager.cc:1397] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 02:58:01.857625 12373 ts_tablet_manager.cc:1428] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260808 02:58:01.858855 12372 raft_consensus.cc:348] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:01.859267 12372 raft_consensus.cc:374] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:01.859452 12372 raft_consensus.cc:729] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af025c42d7a4c36ac51c6495b535398, State: Initialized, Role: FOLLOWER
I20260808 02:58:01.859862 12372 consensus_queue.cc:260] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:01.860734 12372 ts_tablet_manager.cc:1428] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260808 02:58:01.854230 12376 raft_consensus.cc:348] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:01.861020 12376 raft_consensus.cc:374] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:01.861212 12376 raft_consensus.cc:729] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aabdcbc57d64706bf48c93e177d18f3, State: Initialized, Role: FOLLOWER
I20260808 02:58:01.861976 12413 ts_tablet_manager.cc:1918] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:58:01.862167 12413 log.cc:1196] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/a4f7c7d934654e40a8f8a439eac3348a
I20260808 02:58:01.862802 12413 ts_tablet_manager.cc:1939] T a4f7c7d934654e40a8f8a439eac3348a P 5af025c42d7a4c36ac51c6495b535398: Deleting consensus metadata
I20260808 02:58:01.861838 12376 consensus_queue.cc:260] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:01.865193 12376 ts_tablet_manager.cc:1428] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3: Time spent starting tablet: real 0.012s user 0.001s sys 0.002s
I20260808 02:58:01.866183 11621 catalog_manager.cc:4841] TS 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157): tablet a4f7c7d934654e40a8f8a439eac3348a (table test-table [id=d70684f7888147aeae87627b2bdbf098]) successfully deleted
I20260808 02:58:01.907896 12382 raft_consensus.cc:482] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:58:01.908159 12382 raft_consensus.cc:504] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:01.910002 12382 leader_election.cc:288] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157), 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453)
I20260808 02:58:01.919842 12371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.911240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:38501 (local address 127.0.205.130:46157)
0808 02:58:01.911473 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:01.911482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:01.912174 (+ 692us) server_negotiation.cc:407] Connection header received
0808 02:58:01.912563 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:01.912570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:01.912648 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:01.912753 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:01.913447 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.914637 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.917459 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.918470 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.918662 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.918741 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:01.919353 (+ 612us) server_negotiation.cc:299] Negotiation successful
0808 02:58:01.919502 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 02:58:01.920185 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.910728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:53289 (local address 127.0.205.131:35453)
0808 02:58:01.911018 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:01.911027 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:01.912456 (+ 1429us) server_negotiation.cc:407] Connection header received
0808 02:58:01.912596 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:01.912602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:01.912717 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:01.912824 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:01.913624 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.914474 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.917421 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.918826 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.919008 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.919086 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:01.919686 (+ 600us) server_negotiation.cc:299] Negotiation successful
0808 02:58:01.919867 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 02:58:01.921007 12419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.910345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:46157 (local address 127.0.205.129:38501)
0808 02:58:01.912028 (+ 1683us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:01.912054 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:01.912331 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:01.912776 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:01.912781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:01.912807 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:01.913212 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.913228 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.914805 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.914810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.917220 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.917239 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.918932 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.918939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.919127 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.919163 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:58:01.920551 (+ 1388us) client_negotiation.cc:240] Negotiation successful
0808 02:58:01.920667 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1510,"mutex_wait_us":86,"thread_start_us":1703,"threads_started":1}
I20260808 02:58:01.921059 12418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.910503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:35453 (local address 127.0.205.129:53289)
0808 02:58:01.912342 (+ 1839us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:01.912358 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:01.912469 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:01.912981 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:01.912990 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:01.913011 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:01.913456 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.913473 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.914622 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.914630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.917220 (+ 2590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.917239 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.919283 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.919290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.919478 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.919515 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:58:01.920390 (+ 875us) client_negotiation.cc:240] Negotiation successful
0808 02:58:01.920551 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1715,"mutex_wait_us":79,"thread_start_us":1176,"threads_started":1}
I20260808 02:58:01.921733 12071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2a3ead846dd435f8fdec9a7add584ca" candidate_uuid: "8abc5118a51540278c04093762b87151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af025c42d7a4c36ac51c6495b535398" is_pre_election: true
I20260808 02:58:01.922067 12071 raft_consensus.cc:2454] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8abc5118a51540278c04093762b87151 in term 0.
I20260808 02:58:01.922150 12270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2a3ead846dd435f8fdec9a7add584ca" candidate_uuid: "8abc5118a51540278c04093762b87151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aabdcbc57d64706bf48c93e177d18f3" is_pre_election: true
I20260808 02:58:01.922595 11774 leader_election.cc:302] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af025c42d7a4c36ac51c6495b535398, 8abc5118a51540278c04093762b87151; no voters:
I20260808 02:58:01.922665 12270 raft_consensus.cc:2454] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8abc5118a51540278c04093762b87151 in term 0.
I20260808 02:58:01.923012 12382 raft_consensus.cc:2781] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:58:01.923161 12382 raft_consensus.cc:482] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:58:01.923264 12382 raft_consensus.cc:3037] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:01.925504 12382 raft_consensus.cc:504] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:01.926263 12382 leader_election.cc:288] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [CANDIDATE]: Term 1 election: Requested vote from peers 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157), 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453)
I20260808 02:58:01.926752 12071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2a3ead846dd435f8fdec9a7add584ca" candidate_uuid: "8abc5118a51540278c04093762b87151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af025c42d7a4c36ac51c6495b535398"
I20260808 02:58:01.926810 12270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2a3ead846dd435f8fdec9a7add584ca" candidate_uuid: "8abc5118a51540278c04093762b87151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aabdcbc57d64706bf48c93e177d18f3"
I20260808 02:58:01.927017 12270 raft_consensus.cc:3037] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:01.927003 12071 raft_consensus.cc:3037] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:01.930212 12071 raft_consensus.cc:2454] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8abc5118a51540278c04093762b87151 in term 1.
I20260808 02:58:01.930476 12270 raft_consensus.cc:2454] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8abc5118a51540278c04093762b87151 in term 1.
I20260808 02:58:01.930805 11774 leader_election.cc:302] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af025c42d7a4c36ac51c6495b535398, 8abc5118a51540278c04093762b87151; no voters:
I20260808 02:58:01.931269 12382 raft_consensus.cc:2781] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:58:01.931788 12382 raft_consensus.cc:686] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 1 LEADER]: Becoming Leader. State: Replica: 8abc5118a51540278c04093762b87151, State: Running, Role: LEADER
I20260808 02:58:01.932227 12382 consensus_queue.cc:237] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:01.935798 11635 catalog_manager.cc:5495] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8abc5118a51540278c04093762b87151 (127.0.205.129). New cstate: current_term: 1 leader_uuid: "8abc5118a51540278c04093762b87151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:58:01 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157878, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.190@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:58:01.990034 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.972828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43491 (local address 127.0.0.1:52584)
0808 02:58:01.973829 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:01.973850 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:01.974059 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:01.974427 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:01.974430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:01.975207 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:01.975479 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.975488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.977071 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.977075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.978275 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:01.978288 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.979208 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.979212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:01.979343 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.980101 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:01.980131 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:01.983173 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:01.987019 (+ 3846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:01.987033 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:01.987041 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:01.987605 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:01.988070 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:01.988077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:01.988082 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:01.988264 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:01.988837 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:01.988846 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:01.989042 (+ 196us) client_negotiation.cc:769] Sending connection context
0808 02:58:01.989229 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 02:58:01.989537 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"thread_start_us":684,"threads_started":1}
I20260808 02:58:01.990200 12428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:01.972816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52584 (local address 127.0.205.190:43491)
0808 02:58:01.973621 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:01.973630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:01.974002 (+ 372us) server_negotiation.cc:407] Connection header received
0808 02:58:01.974179 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:01.974184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:01.974267 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:01.974389 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:01.975674 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.976932 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.978413 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:01.979052 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:01.979175 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:01.983386 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:01.983392 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:01.983397 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:01.983433 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:01.986865 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:01.987750 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:01.987756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:01.987760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:01.987953 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:01.988381 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:01.988385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:01.988389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:01.988704 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:01.988840 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:01.989363 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 02:58:01.989710 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":598,"thread_start_us":344,"threads_started":1}
I20260808 02:58:01.996476 11636 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:52584:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:58:01.997750 11636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:58:02.005602 11838 tablet_service.cc:1467] Processing CreateTablet for tablet 47abaed6c18041e787644f83bcbe5ff2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f15a38d9fa5845edab5c0160024714d8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:58:02.005918 12042 tablet_service.cc:1467] Processing CreateTablet for tablet 47abaed6c18041e787644f83bcbe5ff2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f15a38d9fa5845edab5c0160024714d8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:58:02.006256 11838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47abaed6c18041e787644f83bcbe5ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:02.006152 12238 tablet_service.cc:1467] Processing CreateTablet for tablet 47abaed6c18041e787644f83bcbe5ff2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f15a38d9fa5845edab5c0160024714d8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:58:02.006518 12042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47abaed6c18041e787644f83bcbe5ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:02.006727 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47abaed6c18041e787644f83bcbe5ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:02.014603 12372 tablet_bootstrap.cc:492] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: Bootstrap starting.
I20260808 02:58:02.014607 12376 tablet_bootstrap.cc:492] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: Bootstrap starting.
I20260808 02:58:02.016499 12376 tablet_bootstrap.cc:654] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:02.016494 12372 tablet_bootstrap.cc:654] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:02.017732 12373 tablet_bootstrap.cc:492] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: Bootstrap starting.
I20260808 02:58:02.019428 12376 tablet_bootstrap.cc:492] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: No bootstrap required, opened a new log
I20260808 02:58:02.019582 12373 tablet_bootstrap.cc:654] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:02.019616 12376 ts_tablet_manager.cc:1397] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 02:58:02.020448 12376 raft_consensus.cc:348] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.020710 12376 raft_consensus.cc:374] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:02.020794 12376 raft_consensus.cc:729] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aabdcbc57d64706bf48c93e177d18f3, State: Initialized, Role: FOLLOWER
I20260808 02:58:02.021072 12376 consensus_queue.cc:260] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.021720 12383 tablet_replica.cc:439] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } }
I20260808 02:58:02.021934 12376 ts_tablet_manager.cc:1428] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 02:58:02.022076 12383 tablet_replica.cc:442] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:58:02.022907 12372 tablet_bootstrap.cc:492] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: No bootstrap required, opened a new log
I20260808 02:58:02.023087 12372 ts_tablet_manager.cc:1397] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.001s
I20260808 02:58:02.023874 12372 raft_consensus.cc:348] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.024125 12372 raft_consensus.cc:374] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:02.024209 12372 raft_consensus.cc:729] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af025c42d7a4c36ac51c6495b535398, State: Initialized, Role: FOLLOWER
I20260808 02:58:02.024463 12372 consensus_queue.cc:260] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.024981 12380 tablet_replica.cc:439] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } }
I20260808 02:58:02.025316 12380 tablet_replica.cc:442] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:58:02.026357 12372 ts_tablet_manager.cc:1428] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 02:58:02.027995 12373 tablet_bootstrap.cc:492] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: No bootstrap required, opened a new log
I20260808 02:58:02.028167 12373 ts_tablet_manager.cc:1397] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260808 02:58:02.028978 12373 raft_consensus.cc:348] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.029234 12373 raft_consensus.cc:374] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:02.029318 12373 raft_consensus.cc:729] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8abc5118a51540278c04093762b87151, State: Initialized, Role: FOLLOWER
I20260808 02:58:02.029691 12373 consensus_queue.cc:260] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.030256 12382 tablet_replica.cc:439] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } }
I20260808 02:58:02.030488 12373 ts_tablet_manager.cc:1428] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:58:02.030582 12382 tablet_replica.cc:442] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:58:02.049577 12383 raft_consensus.cc:482] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:58:02.049798 12383 raft_consensus.cc:504] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.050637 12383 leader_election.cc:288] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8abc5118a51540278c04093762b87151 (127.0.205.129:43055), 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157)
I20260808 02:58:02.051153 11880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47abaed6c18041e787644f83bcbe5ff2" candidate_uuid: "1aabdcbc57d64706bf48c93e177d18f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8abc5118a51540278c04093762b87151" is_pre_election: true
I20260808 02:58:02.051436 11880 raft_consensus.cc:2454] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aabdcbc57d64706bf48c93e177d18f3 in term 0.
I20260808 02:58:02.051370 12071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47abaed6c18041e787644f83bcbe5ff2" candidate_uuid: "1aabdcbc57d64706bf48c93e177d18f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af025c42d7a4c36ac51c6495b535398" is_pre_election: true
I20260808 02:58:02.051671 12071 raft_consensus.cc:2454] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aabdcbc57d64706bf48c93e177d18f3 in term 0.
I20260808 02:58:02.051890 12173 leader_election.cc:302] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aabdcbc57d64706bf48c93e177d18f3, 8abc5118a51540278c04093762b87151; no voters:
I20260808 02:58:02.052183 12383 raft_consensus.cc:2781] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:58:02.052341 12383 raft_consensus.cc:482] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:58:02.052455 12383 raft_consensus.cc:3037] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:02.054852 12383 raft_consensus.cc:504] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.055413 12383 leader_election.cc:288] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [CANDIDATE]: Term 1 election: Requested vote from peers 8abc5118a51540278c04093762b87151 (127.0.205.129:43055), 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157)
I20260808 02:58:02.055960 11880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47abaed6c18041e787644f83bcbe5ff2" candidate_uuid: "1aabdcbc57d64706bf48c93e177d18f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8abc5118a51540278c04093762b87151"
I20260808 02:58:02.055986 12071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47abaed6c18041e787644f83bcbe5ff2" candidate_uuid: "1aabdcbc57d64706bf48c93e177d18f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af025c42d7a4c36ac51c6495b535398"
I20260808 02:58:02.056170 11880 raft_consensus.cc:3037] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:02.056236 12071 raft_consensus.cc:3037] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:02.058935 11880 raft_consensus.cc:2454] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aabdcbc57d64706bf48c93e177d18f3 in term 1.
I20260808 02:58:02.059422 12173 leader_election.cc:302] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aabdcbc57d64706bf48c93e177d18f3, 8abc5118a51540278c04093762b87151; no voters:
I20260808 02:58:02.059535 12071 raft_consensus.cc:2454] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aabdcbc57d64706bf48c93e177d18f3 in term 1.
I20260808 02:58:02.059813 12383 raft_consensus.cc:2781] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:58:02.060035 12383 raft_consensus.cc:686] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 LEADER]: Becoming Leader. State: Replica: 1aabdcbc57d64706bf48c93e177d18f3, State: Running, Role: LEADER
I20260808 02:58:02.060349 12383 consensus_queue.cc:237] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.062350 12403 tablet_replica.cc:439] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: TxnStatusTablet state changed. Reason: New leader 1aabdcbc57d64706bf48c93e177d18f3. Latest consensus state: current_term: 1 leader_uuid: "1aabdcbc57d64706bf48c93e177d18f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } }
I20260808 02:58:02.062836 12403 tablet_replica.cc:442] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:58:02.063694 12434 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:58:02.063943 12434 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:58:02.063938 11637 catalog_manager.cc:5495] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131). New cstate: current_term: 1 leader_uuid: "1aabdcbc57d64706bf48c93e177d18f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } health_report { overall_health: HEALTHY } } }
I20260808 02:58:02.091420 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.085536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:35453 (local address 127.0.0.1:43312)
0808 02:58:02.085751 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:02.085770 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:02.085949 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:02.086370 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:02.086373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:02.086384 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:58:02.086575 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.086581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.088064 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.088067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.089076 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.089088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.090181 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.090186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.090369 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.090441 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.090891 (+ 450us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.090911 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:58:02.090981 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:58:02.091109 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 02:58:02.091416 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.085593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.0.205.131:35453)
0808 02:58:02.085804 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.085809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.085830 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:58:02.086144 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.086151 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.086236 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.086329 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:58:02.086714 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.087926 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.089227 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.089973 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.090109 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.090642 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.090791 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.090889 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.091007 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.091092 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 02:58:02.095669 12071 raft_consensus.cc:1261] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Refusing update from remote peer 1aabdcbc57d64706bf48c93e177d18f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:02.095734 11880 raft_consensus.cc:1261] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Refusing update from remote peer 1aabdcbc57d64706bf48c93e177d18f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:02.096314 12383 consensus_queue.cc:1035] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:02.096648 12403 consensus_queue.cc:1035] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:02.100351 12380 tablet_replica.cc:439] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: TxnStatusTablet state changed. Reason: New leader 1aabdcbc57d64706bf48c93e177d18f3. Latest consensus state: current_term: 1 leader_uuid: "1aabdcbc57d64706bf48c93e177d18f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } }
I20260808 02:58:02.100626 12380 tablet_replica.cc:442] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:58:02.102515 12382 tablet_replica.cc:439] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: TxnStatusTablet state changed. Reason: New leader 1aabdcbc57d64706bf48c93e177d18f3. Latest consensus state: current_term: 1 leader_uuid: "1aabdcbc57d64706bf48c93e177d18f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } }
I20260808 02:58:02.102857 12382 tablet_replica.cc:442] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:58:02.104496 12386 tablet_replica.cc:439] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1aabdcbc57d64706bf48c93e177d18f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } }
I20260808 02:58:02.104892 12386 tablet_replica.cc:442] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:58:02.105082 12403 tablet_replica.cc:439] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1aabdcbc57d64706bf48c93e177d18f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } }
I20260808 02:58:02.105343 12403 tablet_replica.cc:442] T 47abaed6c18041e787644f83bcbe5ff2 P 1aabdcbc57d64706bf48c93e177d18f3: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:58:02.106681 12398 mvcc.cc:205] Tried to move back new op lower bound from 7316102685056712704 to 7316102684921962496. Current Snapshot: MvccSnapshot[applied={T|T < 7316102685056712704}]
I20260808 02:58:02.107280 12380 tablet_replica.cc:439] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1aabdcbc57d64706bf48c93e177d18f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } }
I20260808 02:58:02.107528 12380 tablet_replica.cc:442] T 47abaed6c18041e787644f83bcbe5ff2 P 5af025c42d7a4c36ac51c6495b535398: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:58:02.112648 12382 tablet_replica.cc:439] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1aabdcbc57d64706bf48c93e177d18f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } } }
I20260808 02:58:02.113018 12382 tablet_replica.cc:442] T 47abaed6c18041e787644f83bcbe5ff2 P 8abc5118a51540278c04093762b87151: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:58:02.119201 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.113329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.0.205.129:43055)
0808 02:58:02.113503 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.113506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.113521 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:58:02.113646 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.113650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.113781 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.113981 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:58:02.114473 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.115323 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.116875 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.117287 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.117436 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.117957 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.118541 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.118704 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.118787 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.118875 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 02:58:02.142077 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.132231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:35005 (local address 127.0.205.131:35453)
0808 02:58:02.132485 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.132492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.134614 (+ 2122us) server_negotiation.cc:407] Connection header received
0808 02:58:02.134832 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.134838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.134932 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.135031 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:58:02.135651 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.136913 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.139250 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.140352 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.140487 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.141103 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.141307 (+ 204us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.141415 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.141560 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.141673 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:58:02.142197 12445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.132248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:35453 (local address 127.0.205.129:35005)
0808 02:58:02.134474 (+ 2226us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:02.134498 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:02.134682 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:02.135075 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:02.135080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:02.135095 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:58:02.135467 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.135479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.137076 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.137083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.138959 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.138976 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.140588 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.140595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.140800 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.140885 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.141421 (+ 536us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.141432 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 02:58:02.141512 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 02:58:02.141677 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2029,"thread_start_us":1837,"threads_started":1}
I20260808 02:58:02.154529 12270 raft_consensus.cc:1261] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Refusing update from remote peer 8abc5118a51540278c04093762b87151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:02.154606 12071 raft_consensus.cc:1261] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Refusing update from remote peer 8abc5118a51540278c04093762b87151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:02.155402 12420 consensus_queue.cc:1035] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:02.156252 12420 consensus_queue.cc:1035] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:02.203950 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.196976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:51273 (local address 127.0.205.129:43055)
0808 02:58:02.197373 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.197381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.197754 (+ 373us) server_negotiation.cc:407] Connection header received
0808 02:58:02.198115 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.198121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.198218 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.198317 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:58:02.198825 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.199994 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.201790 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.202479 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.202623 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.203059 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.203163 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.203260 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.203556 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.203633 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260808 02:58:02.203986 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.196956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43055 (local address 127.0.205.131:51273)
0808 02:58:02.197647 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:02.197669 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:02.197924 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:02.198351 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:02.198355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:02.198368 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:58:02.198660 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.198671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.200114 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.200118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.201617 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.201630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.202658 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.202667 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.202796 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.202860 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.203262 (+ 402us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.203277 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 02:58:02.203379 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 02:58:02.203554 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":690,"threads_started":1}
I20260808 02:58:02.248490 11636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:58:02.248797 11636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:58:02.252764 11636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2: Sending DeleteTablet for 3 replicas of tablet b2a3ead846dd435f8fdec9a7add584ca
I20260808 02:58:02.253958 11838 tablet_service.cc:1514] Processing DeleteTablet for tablet b2a3ead846dd435f8fdec9a7add584ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:02 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:46162
I20260808 02:58:02.254088 12042 tablet_service.cc:1514] Processing DeleteTablet for tablet b2a3ead846dd435f8fdec9a7add584ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:02 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:34682
I20260808 02:58:02.254315 12411 tablet_replica.cc:330] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151: stopping tablet replica
I20260808 02:58:02.254572 12411 raft_consensus.cc:2229] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:58:02.255193 12411 raft_consensus.cc:2258] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:02.255553 12413 tablet_replica.cc:330] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398: stopping tablet replica
I20260808 02:58:02.255864 12413 raft_consensus.cc:2229] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:58:02.256096 12413 raft_consensus.cc:2258] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:02.257807 12413 ts_tablet_manager.cc:1905] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:02.262805 12411 ts_tablet_manager.cc:1905] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:02.266192 12413 ts_tablet_manager.cc:1918] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:58:02.266501 12413 log.cc:1196] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/b2a3ead846dd435f8fdec9a7add584ca
I20260808 02:58:02.266471 12238 tablet_service.cc:1514] Processing DeleteTablet for tablet b2a3ead846dd435f8fdec9a7add584ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:02 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:43288
I20260808 02:58:02.266911 12412 tablet_replica.cc:330] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3: stopping tablet replica
I20260808 02:58:02.267163 12412 raft_consensus.cc:2229] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:58:02.267427 12412 raft_consensus.cc:2258] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:02.270334 12412 ts_tablet_manager.cc:1905] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:02.278848 12412 ts_tablet_manager.cc:1918] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:58:02.280154 12413 ts_tablet_manager.cc:1939] T b2a3ead846dd435f8fdec9a7add584ca P 5af025c42d7a4c36ac51c6495b535398: Deleting consensus metadata
I20260808 02:58:02.281757 11621 catalog_manager.cc:4841] TS 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157): tablet b2a3ead846dd435f8fdec9a7add584ca (table test-table [id=e1fd4274a862414ba04f23e83d112228]) successfully deleted
I20260808 02:58:02.282341 12412 log.cc:1196] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/b2a3ead846dd435f8fdec9a7add584ca
I20260808 02:58:02.283016 12412 ts_tablet_manager.cc:1939] T b2a3ead846dd435f8fdec9a7add584ca P 1aabdcbc57d64706bf48c93e177d18f3: Deleting consensus metadata
I20260808 02:58:02.284545 11622 catalog_manager.cc:4841] TS 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453): tablet b2a3ead846dd435f8fdec9a7add584ca (table test-table [id=e1fd4274a862414ba04f23e83d112228]) successfully deleted
I20260808 02:58:02.287427 12411 ts_tablet_manager.cc:1918] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:58:02.287703 12411 log.cc:1196] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/b2a3ead846dd435f8fdec9a7add584ca
I20260808 02:58:02.288349 12411 ts_tablet_manager.cc:1939] T b2a3ead846dd435f8fdec9a7add584ca P 8abc5118a51540278c04093762b87151: Deleting consensus metadata
I20260808 02:58:02.289651 11623 catalog_manager.cc:4841] TS 8abc5118a51540278c04093762b87151 (127.0.205.129:43055): tablet b2a3ead846dd435f8fdec9a7add584ca (table test-table [id=e1fd4274a862414ba04f23e83d112228]) successfully deleted
Aug 08 02:58:02 dist-test-slave-6ntk krb5kdc[11579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:58:02 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:02 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:58:02.366130 12428 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:02.351524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.0.205.190:43491)
0808 02:58:02.351788 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.351795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.352144 (+ 349us) server_negotiation.cc:407] Connection header received
0808 02:58:02.352388 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.352394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.352481 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.352595 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:02.354479 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.355669 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.357623 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.358410 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.358552 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.365809 (+ 7257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52598: BlockingRecv error: recv got EOF from 127.0.0.1:52598 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 02:58:02 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM
I20260808 02:58:02.395015 12428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.375185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.0.205.190:43491)
0808 02:58:02.375392 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.375399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.375764 (+ 365us) server_negotiation.cc:407] Connection header received
0808 02:58:02.375983 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.375989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.376074 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.376184 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:02.377958 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.379133 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.381095 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.381797 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.381994 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.387475 (+ 5481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:02.387484 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:02.387490 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:02.387561 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:02.391606 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:02.392545 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:02.392552 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:02.392556 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:02.392758 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:02.393211 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:02.393217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:02.393221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:02.393613 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:02.393969 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.394286 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.394547 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 02:58:02.399492 11636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:58:02.400280 11636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:58:02.409009 12042 tablet_service.cc:1467] Processing CreateTablet for tablet 7fddc40e59304f718a1bbcd03ced0ee1 (DEFAULT_TABLE table=test-table [id=8d5caf959a0f40f2a7fdf062f261afbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:02.409080 12238 tablet_service.cc:1467] Processing CreateTablet for tablet 7fddc40e59304f718a1bbcd03ced0ee1 (DEFAULT_TABLE table=test-table [id=8d5caf959a0f40f2a7fdf062f261afbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:02.409571 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fddc40e59304f718a1bbcd03ced0ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:02.409570 12042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fddc40e59304f718a1bbcd03ced0ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:02.415350 11838 tablet_service.cc:1467] Processing CreateTablet for tablet 7fddc40e59304f718a1bbcd03ced0ee1 (DEFAULT_TABLE table=test-table [id=8d5caf959a0f40f2a7fdf062f261afbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:02.415851 11838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fddc40e59304f718a1bbcd03ced0ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:02.417488 12372 tablet_bootstrap.cc:492] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398: Bootstrap starting.
I20260808 02:58:02.419387 12372 tablet_bootstrap.cc:654] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:02.425961 12376 tablet_bootstrap.cc:492] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3: Bootstrap starting.
I20260808 02:58:02.427454 12373 tablet_bootstrap.cc:492] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151: Bootstrap starting.
I20260808 02:58:02.427798 12376 tablet_bootstrap.cc:654] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:02.429376 12373 tablet_bootstrap.cc:654] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:02.430271 12376 tablet_bootstrap.cc:492] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3: No bootstrap required, opened a new log
I20260808 02:58:02.430491 12376 ts_tablet_manager.cc:1397] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 02:58:02.431442 12376 raft_consensus.cc:348] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.431885 12376 raft_consensus.cc:374] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:02.432039 12376 raft_consensus.cc:729] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aabdcbc57d64706bf48c93e177d18f3, State: Initialized, Role: FOLLOWER
I20260808 02:58:02.432341 12376 consensus_queue.cc:260] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.435477 12376 ts_tablet_manager.cc:1428] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260808 02:58:02.439746 12373 tablet_bootstrap.cc:492] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151: No bootstrap required, opened a new log
I20260808 02:58:02.440137 12373 ts_tablet_manager.cc:1397] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.003s
I20260808 02:58:02.441087 12373 raft_consensus.cc:348] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.441431 12373 raft_consensus.cc:374] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:02.441548 12373 raft_consensus.cc:729] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8abc5118a51540278c04093762b87151, State: Initialized, Role: FOLLOWER
I20260808 02:58:02.441834 12373 consensus_queue.cc:260] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.443066 12372 tablet_bootstrap.cc:492] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398: No bootstrap required, opened a new log
I20260808 02:58:02.443274 12372 ts_tablet_manager.cc:1397] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.011s
I20260808 02:58:02.444137 12372 raft_consensus.cc:348] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.444455 12372 raft_consensus.cc:374] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:02.444576 12372 raft_consensus.cc:729] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af025c42d7a4c36ac51c6495b535398, State: Initialized, Role: FOLLOWER
I20260808 02:58:02.444824 12372 consensus_queue.cc:260] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.445207 12456 raft_consensus.cc:482] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:58:02.445477 12372 ts_tablet_manager.cc:1428] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:58:02.445407 12456 raft_consensus.cc:504] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.446270 12456 leader_election.cc:288] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157), 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453)
I20260808 02:58:02.446907 12071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fddc40e59304f718a1bbcd03ced0ee1" candidate_uuid: "8abc5118a51540278c04093762b87151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af025c42d7a4c36ac51c6495b535398" is_pre_election: true
I20260808 02:58:02.447146 12071 raft_consensus.cc:2454] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8abc5118a51540278c04093762b87151 in term 0.
I20260808 02:58:02.447149 12269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fddc40e59304f718a1bbcd03ced0ee1" candidate_uuid: "8abc5118a51540278c04093762b87151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aabdcbc57d64706bf48c93e177d18f3" is_pre_election: true
I20260808 02:58:02.447407 12269 raft_consensus.cc:2454] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8abc5118a51540278c04093762b87151 in term 0.
I20260808 02:58:02.447707 11774 leader_election.cc:302] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af025c42d7a4c36ac51c6495b535398, 8abc5118a51540278c04093762b87151; no voters:
I20260808 02:58:02.448012 12456 raft_consensus.cc:2781] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:58:02.448143 12456 raft_consensus.cc:482] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:58:02.448237 12456 raft_consensus.cc:3037] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:02.451581 12456 raft_consensus.cc:504] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.452253 12456 leader_election.cc:288] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [CANDIDATE]: Term 1 election: Requested vote from peers 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157), 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453)
I20260808 02:58:02.453001 12269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fddc40e59304f718a1bbcd03ced0ee1" candidate_uuid: "8abc5118a51540278c04093762b87151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aabdcbc57d64706bf48c93e177d18f3"
I20260808 02:58:02.453219 12269 raft_consensus.cc:3037] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:02.453207 12071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fddc40e59304f718a1bbcd03ced0ee1" candidate_uuid: "8abc5118a51540278c04093762b87151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af025c42d7a4c36ac51c6495b535398"
I20260808 02:58:02.453413 12071 raft_consensus.cc:3037] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:02.456506 12269 raft_consensus.cc:2454] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8abc5118a51540278c04093762b87151 in term 1.
I20260808 02:58:02.457839 12071 raft_consensus.cc:2454] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8abc5118a51540278c04093762b87151 in term 1.
I20260808 02:58:02.458099 11775 leader_election.cc:302] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aabdcbc57d64706bf48c93e177d18f3, 8abc5118a51540278c04093762b87151; no voters:
I20260808 02:58:02.458866 12456 raft_consensus.cc:2781] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:58:02.459100 12456 raft_consensus.cc:686] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 1 LEADER]: Becoming Leader. State: Replica: 8abc5118a51540278c04093762b87151, State: Running, Role: LEADER
I20260808 02:58:02.460155 12456 consensus_queue.cc:237] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.461833 12373 ts_tablet_manager.cc:1428] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20260808 02:58:02.464452 11636 catalog_manager.cc:5495] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8abc5118a51540278c04093762b87151 (127.0.205.129). New cstate: current_term: 1 leader_uuid: "8abc5118a51540278c04093762b87151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } health_report { overall_health: UNKNOWN } } }
I20260808 02:58:02.484486 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.477494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46182 (local address 127.0.205.129:43055)
0808 02:58:02.477708 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.477715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.477963 (+ 248us) server_negotiation.cc:407] Connection header received
0808 02:58:02.478191 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.478197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.478280 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.478381 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:58:02.478852 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.480015 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.482156 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.482902 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.483043 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.483610 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.483763 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.483866 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.484028 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.484124 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 02:58:02.487636 12269 raft_consensus.cc:1261] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Refusing update from remote peer 8abc5118a51540278c04093762b87151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:02.488353 12071 raft_consensus.cc:1261] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Refusing update from remote peer 8abc5118a51540278c04093762b87151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:02.488355 12456 consensus_queue.cc:1035] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:02.489164 12452 consensus_queue.cc:1035] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:02.501477 11636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:58:02.501744 11636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:58:02.505746 11636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2: Sending DeleteTablet for 3 replicas of tablet 7fddc40e59304f718a1bbcd03ced0ee1
I20260808 02:58:02.507267 11838 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fddc40e59304f718a1bbcd03ced0ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:02 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:46162
I20260808 02:58:02.508015 12411 tablet_replica.cc:330] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151: stopping tablet replica
I20260808 02:58:02.508256 12411 raft_consensus.cc:2229] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:58:02.508641 12411 raft_consensus.cc:2258] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:02.509680 12238 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fddc40e59304f718a1bbcd03ced0ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:02 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:43288
I20260808 02:58:02.510015 12412 tablet_replica.cc:330] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3: stopping tablet replica
I20260808 02:58:02.510227 12412 raft_consensus.cc:2229] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:58:02.510490 12412 raft_consensus.cc:2258] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:02.511716 12042 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fddc40e59304f718a1bbcd03ced0ee1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:02 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:34682
I20260808 02:58:02.512108 12413 tablet_replica.cc:330] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398: stopping tablet replica
I20260808 02:58:02.512308 12413 raft_consensus.cc:2229] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:58:02.512140 11637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 02:58:02.512562 12413 raft_consensus.cc:2258] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 02:58:02.512859 11637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:58:02.514926 12411 ts_tablet_manager.cc:1905] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:02.521134 11838 tablet_service.cc:1467] Processing CreateTablet for tablet 332a581a9a7d44c28f22b78e4b25baec (DEFAULT_TABLE table=test-table [id=708d82ed24704ca6a02a80531914062c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:02.521580 11838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332a581a9a7d44c28f22b78e4b25baec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:02.523038 12411 ts_tablet_manager.cc:1918] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:58:02.523193 12411 log.cc:1196] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/7fddc40e59304f718a1bbcd03ced0ee1
I20260808 02:58:02.523295 12238 tablet_service.cc:1467] Processing CreateTablet for tablet 332a581a9a7d44c28f22b78e4b25baec (DEFAULT_TABLE table=test-table [id=708d82ed24704ca6a02a80531914062c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:02.523721 12411 ts_tablet_manager.cc:1939] T 7fddc40e59304f718a1bbcd03ced0ee1 P 8abc5118a51540278c04093762b87151: Deleting consensus metadata
I20260808 02:58:02.523813 12238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332a581a9a7d44c28f22b78e4b25baec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:02.524852 11623 catalog_manager.cc:4841] TS 8abc5118a51540278c04093762b87151 (127.0.205.129:43055): tablet 7fddc40e59304f718a1bbcd03ced0ee1 (table test-table [id=8d5caf959a0f40f2a7fdf062f261afbd]) successfully deleted
I20260808 02:58:02.526283 12413 ts_tablet_manager.cc:1905] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:02.530241 12042 tablet_service.cc:1467] Processing CreateTablet for tablet 332a581a9a7d44c28f22b78e4b25baec (DEFAULT_TABLE table=test-table [id=708d82ed24704ca6a02a80531914062c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:02.530882 12042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332a581a9a7d44c28f22b78e4b25baec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:02.531646 12373 tablet_bootstrap.cc:492] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151: Bootstrap starting.
I20260808 02:58:02.533432 12373 tablet_bootstrap.cc:654] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:02.536906 12413 ts_tablet_manager.cc:1918] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:58:02.537052 12413 log.cc:1196] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/7fddc40e59304f718a1bbcd03ced0ee1
I20260808 02:58:02.537539 12413 ts_tablet_manager.cc:1939] T 7fddc40e59304f718a1bbcd03ced0ee1 P 5af025c42d7a4c36ac51c6495b535398: Deleting consensus metadata
I20260808 02:58:02.538353 12376 tablet_bootstrap.cc:492] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3: Bootstrap starting.
I20260808 02:58:02.538746 12412 ts_tablet_manager.cc:1905] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:02.540282 12376 tablet_bootstrap.cc:654] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:02.542824 12372 tablet_bootstrap.cc:492] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398: Bootstrap starting.
I20260808 02:58:02.544593 12372 tablet_bootstrap.cc:654] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:02.546396 12412 ts_tablet_manager.cc:1918] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:58:02.546684 12412 log.cc:1196] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/7fddc40e59304f718a1bbcd03ced0ee1
I20260808 02:58:02.547330 12412 ts_tablet_manager.cc:1939] T 7fddc40e59304f718a1bbcd03ced0ee1 P 1aabdcbc57d64706bf48c93e177d18f3: Deleting consensus metadata
I20260808 02:58:02.548765 11622 catalog_manager.cc:4841] TS 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453): tablet 7fddc40e59304f718a1bbcd03ced0ee1 (table test-table [id=8d5caf959a0f40f2a7fdf062f261afbd]) successfully deleted
I20260808 02:58:02.549958 11621 catalog_manager.cc:4841] TS 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157): tablet 7fddc40e59304f718a1bbcd03ced0ee1 (table test-table [id=8d5caf959a0f40f2a7fdf062f261afbd]) successfully deleted
I20260808 02:58:02.551867 12373 tablet_bootstrap.cc:492] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151: No bootstrap required, opened a new log
I20260808 02:58:02.552117 12373 ts_tablet_manager.cc:1397] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20260808 02:58:02.552958 12373 raft_consensus.cc:348] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.553154 12376 tablet_bootstrap.cc:492] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3: No bootstrap required, opened a new log
I20260808 02:58:02.553324 12376 ts_tablet_manager.cc:1397] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.006s
I20260808 02:58:02.553309 12373 raft_consensus.cc:374] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:02.553398 12373 raft_consensus.cc:729] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8abc5118a51540278c04093762b87151, State: Initialized, Role: FOLLOWER
I20260808 02:58:02.553735 12373 consensus_queue.cc:260] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.555158 12373 ts_tablet_manager.cc:1428] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 02:58:02.556365 12376 raft_consensus.cc:348] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.556716 12376 raft_consensus.cc:374] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:02.556846 12376 raft_consensus.cc:729] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aabdcbc57d64706bf48c93e177d18f3, State: Initialized, Role: FOLLOWER
I20260808 02:58:02.557142 12376 consensus_queue.cc:260] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.559357 12372 tablet_bootstrap.cc:492] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398: No bootstrap required, opened a new log
I20260808 02:58:02.559525 12372 ts_tablet_manager.cc:1397] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.006s
I20260808 02:58:02.560240 12372 raft_consensus.cc:348] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.560487 12372 raft_consensus.cc:374] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:02.560554 12372 raft_consensus.cc:729] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af025c42d7a4c36ac51c6495b535398, State: Initialized, Role: FOLLOWER
I20260808 02:58:02.560742 12376 ts_tablet_manager.cc:1428] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260808 02:58:02.560770 12372 consensus_queue.cc:260] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.561266 12372 ts_tablet_manager.cc:1428] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 02:58:02.648070 12380 raft_consensus.cc:482] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:58:02.648334 12380 raft_consensus.cc:504] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.650058 12380 leader_election.cc:288] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8abc5118a51540278c04093762b87151 (127.0.205.129:43055), 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453)
I20260808 02:58:02.673367 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.650906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:54503 (local address 127.0.205.129:43055)
0808 02:58:02.651141 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.651147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.663543 (+ 12396us) server_negotiation.cc:407] Connection header received
0808 02:58:02.663812 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.663825 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.663931 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.664058 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:02.664667 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.665824 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.669321 (+ 3497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.670727 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.670893 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.670973 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.672884 (+ 1911us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.673058 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260808 02:58:02.674122 12487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.650680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:35453 (local address 127.0.205.130:47755)
0808 02:58:02.664478 (+ 13798us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:02.664501 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:02.664638 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:02.665246 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:02.665252 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:02.665280 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:02.665691 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.665705 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.667431 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.667437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.669690 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.669706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.673194 (+ 3488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.673201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.673395 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.673429 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 02:58:02.673500 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:58:02.673640 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13636,"mutex_wait_us":64,"thread_start_us":13037,"threads_started":1}
I20260808 02:58:02.674536 12485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.650423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43055 (local address 127.0.205.130:54503)
0808 02:58:02.663409 (+ 12986us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:02.663433 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:02.663617 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:02.664093 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:02.664099 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:02.664127 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:02.664470 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.664483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.666067 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.666072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.669097 (+ 3025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.669115 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.671149 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.671154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.672662 (+ 1508us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.672702 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 02:58:02.673924 (+ 1222us) client_negotiation.cc:240] Negotiation successful
0808 02:58:02.674084 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12840,"mutex_wait_us":31,"thread_start_us":12677,"threads_started":1}
I20260808 02:58:02.676127 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.651355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:47755 (local address 127.0.205.131:35453)
0808 02:58:02.661465 (+ 10110us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.661472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.664770 (+ 3298us) server_negotiation.cc:407] Connection header received
0808 02:58:02.664857 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.664872 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.664953 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.665056 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:02.665962 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.667245 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.669935 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.671279 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.675400 (+ 4121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.675481 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.675539 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.675677 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9953,"thread_start_us":9764,"threads_started":1}
I20260808 02:58:02.676162 11879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332a581a9a7d44c28f22b78e4b25baec" candidate_uuid: "5af025c42d7a4c36ac51c6495b535398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8abc5118a51540278c04093762b87151" is_pre_election: true
I20260808 02:58:02.676507 11879 raft_consensus.cc:2454] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af025c42d7a4c36ac51c6495b535398 in term 0.
I20260808 02:58:02.676946 12269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332a581a9a7d44c28f22b78e4b25baec" candidate_uuid: "5af025c42d7a4c36ac51c6495b535398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aabdcbc57d64706bf48c93e177d18f3" is_pre_election: true
I20260808 02:58:02.677222 12269 raft_consensus.cc:2454] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af025c42d7a4c36ac51c6495b535398 in term 0.
I20260808 02:58:02.677148 11974 leader_election.cc:302] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af025c42d7a4c36ac51c6495b535398, 8abc5118a51540278c04093762b87151; no voters:
I20260808 02:58:02.677640 12380 raft_consensus.cc:2781] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:58:02.677760 12380 raft_consensus.cc:482] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:58:02.677861 12380 raft_consensus.cc:3037] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:02.681248 12380 raft_consensus.cc:504] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.682039 12380 leader_election.cc:288] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [CANDIDATE]: Term 1 election: Requested vote from peers 8abc5118a51540278c04093762b87151 (127.0.205.129:43055), 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453)
I20260808 02:58:02.682541 11879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332a581a9a7d44c28f22b78e4b25baec" candidate_uuid: "5af025c42d7a4c36ac51c6495b535398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8abc5118a51540278c04093762b87151"
I20260808 02:58:02.682693 12269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332a581a9a7d44c28f22b78e4b25baec" candidate_uuid: "5af025c42d7a4c36ac51c6495b535398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aabdcbc57d64706bf48c93e177d18f3"
I20260808 02:58:02.682796 11879 raft_consensus.cc:3037] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:02.682906 12269 raft_consensus.cc:3037] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:02.686129 11879 raft_consensus.cc:2454] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af025c42d7a4c36ac51c6495b535398 in term 1.
I20260808 02:58:02.686163 12269 raft_consensus.cc:2454] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af025c42d7a4c36ac51c6495b535398 in term 1.
I20260808 02:58:02.686636 11974 leader_election.cc:302] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af025c42d7a4c36ac51c6495b535398, 8abc5118a51540278c04093762b87151; no voters:
I20260808 02:58:02.687070 12380 raft_consensus.cc:2781] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:58:02.687577 12380 raft_consensus.cc:686] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 1 LEADER]: Becoming Leader. State: Replica: 5af025c42d7a4c36ac51c6495b535398, State: Running, Role: LEADER
I20260808 02:58:02.688117 12380 consensus_queue.cc:237] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } }
I20260808 02:58:02.692438 11636 catalog_manager.cc:5495] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130). New cstate: current_term: 1 leader_uuid: "5af025c42d7a4c36ac51c6495b535398" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5af025c42d7a4c36ac51c6495b535398" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 46157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 } health_report { overall_health: UNKNOWN } } }
I20260808 02:58:02.727885 12491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.719796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.0.205.130:46157)
0808 02:58:02.720455 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.720464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.720491 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:58:02.720607 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.720614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.720768 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.720975 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:58:02.721508 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.722770 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.724796 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.725446 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.725585 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.726198 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.727057 (+ 859us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.727214 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.727378 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.727471 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":671,"threads_started":1}
I20260808 02:58:02.748932 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.741086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:39285 (local address 127.0.205.131:35453)
0808 02:58:02.741495 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.741502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.742005 (+ 503us) server_negotiation.cc:407] Connection header received
0808 02:58:02.742290 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.742297 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.742392 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.742503 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:58:02.743131 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.744365 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.746658 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.747329 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.747476 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.748096 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.748251 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.748357 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.748536 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.748623 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260808 02:58:02.749076 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.741105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:35453 (local address 127.0.205.130:39285)
0808 02:58:02.741862 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:02.741911 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:02.742071 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:02.742544 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:02.742551 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:02.742564 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:58:02.742938 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.742952 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.744570 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.744576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.746472 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.746492 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.747537 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.747545 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.747724 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.747803 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.748381 (+ 578us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.748401 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:58:02.748499 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 02:58:02.748656 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":408,"threads_started":1}
I20260808 02:58:02.760363 12269 raft_consensus.cc:1261] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Refusing update from remote peer 5af025c42d7a4c36ac51c6495b535398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:02.760535 11879 raft_consensus.cc:1261] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Refusing update from remote peer 5af025c42d7a4c36ac51c6495b535398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:02.761358 12489 consensus_queue.cc:1035] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8abc5118a51540278c04093762b87151" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:02.762218 12380 consensus_queue.cc:1035] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aabdcbc57d64706bf48c93e177d18f3" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:02.825488 12491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.817725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:39947 (local address 127.0.205.130:46157)
0808 02:58:02.817930 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.817937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.818627 (+ 690us) server_negotiation.cc:407] Connection header received
0808 02:58:02.818923 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.818930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.819015 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.819075 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:58:02.819894 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.821027 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.823219 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.823885 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.823981 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.824710 (+ 729us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.824842 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.824899 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.825106 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.825198 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 02:58:02.826538 12501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.817716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:46157 (local address 127.0.205.131:39947)
0808 02:58:02.818517 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:02.818542 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:02.818759 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:02.819276 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:02.819280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:02.819292 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:58:02.819647 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.819657 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.821213 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.821218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.823046 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:02.823061 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.824253 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.824262 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:02.824460 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.824514 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:58:02.824976 (+ 462us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:58:02.824986 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 02:58:02.825991 (+ 1005us) client_negotiation.cc:240] Negotiation successful
0808 02:58:02.826140 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":480,"threads_started":1}
I20260808 02:58:02.876621 11636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:58:02.876919 11636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:58:02.880712 11636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f3402bc91fa4e81af2afd4a46e21bc2: Sending DeleteTablet for 3 replicas of tablet 332a581a9a7d44c28f22b78e4b25baec
I20260808 02:58:02.881577 11838 tablet_service.cc:1514] Processing DeleteTablet for tablet 332a581a9a7d44c28f22b78e4b25baec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:02 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:46162
I20260808 02:58:02.881927 12411 tablet_replica.cc:330] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151: stopping tablet replica
I20260808 02:58:02.882138 12411 raft_consensus.cc:2229] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:58:02.882364 12411 raft_consensus.cc:2258] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:02.882515 12042 tablet_service.cc:1514] Processing DeleteTablet for tablet 332a581a9a7d44c28f22b78e4b25baec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:02 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:34682
I20260808 02:58:02.882838 12413 tablet_replica.cc:330] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398: stopping tablet replica
I20260808 02:58:02.883057 12413 raft_consensus.cc:2229] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:58:02.883585 12413 raft_consensus.cc:2258] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:02.883948 12411 ts_tablet_manager.cc:1905] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:02.886596 12238 tablet_service.cc:1514] Processing DeleteTablet for tablet 332a581a9a7d44c28f22b78e4b25baec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:02 UTC) from {username='oryx', principal='oryx/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:43288
I20260808 02:58:02.886898 12412 tablet_replica.cc:330] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3: stopping tablet replica
I20260808 02:58:02.886983 12413 ts_tablet_manager.cc:1905] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:02.887099 12412 raft_consensus.cc:2229] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:58:02.887292 12412 raft_consensus.cc:2258] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:02.888628 12412 ts_tablet_manager.cc:1905] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:02.894446 12413 ts_tablet_manager.cc:1918] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:58:02.894654 12413 log.cc:1196] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/332a581a9a7d44c28f22b78e4b25baec
I20260808 02:58:02.895123 12413 ts_tablet_manager.cc:1939] T 332a581a9a7d44c28f22b78e4b25baec P 5af025c42d7a4c36ac51c6495b535398: Deleting consensus metadata
I20260808 02:58:02.895713 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 11751
I20260808 02:58:02.898530 11621 catalog_manager.cc:4841] TS 5af025c42d7a4c36ac51c6495b535398 (127.0.205.130:46157): tablet 332a581a9a7d44c28f22b78e4b25baec (table test-table [id=708d82ed24704ca6a02a80531914062c]) successfully deleted
I20260808 02:58:02.904073 12412 ts_tablet_manager.cc:1918] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:58:02.904371 12412 log.cc:1196] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/332a581a9a7d44c28f22b78e4b25baec
I20260808 02:58:02.904909 12412 ts_tablet_manager.cc:1939] T 332a581a9a7d44c28f22b78e4b25baec P 1aabdcbc57d64706bf48c93e177d18f3: Deleting consensus metadata
I20260808 02:58:02.906404 11622 catalog_manager.cc:4841] TS 1aabdcbc57d64706bf48c93e177d18f3 (127.0.205.131:35453): tablet 332a581a9a7d44c28f22b78e4b25baec (table test-table [id=708d82ed24704ca6a02a80531914062c]) successfully deleted
I20260808 02:58:02.908280 12411 ts_tablet_manager.cc:1918] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:58:02.908562 12411 log.cc:1196] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/332a581a9a7d44c28f22b78e4b25baec
I20260808 02:58:02.909092 12411 ts_tablet_manager.cc:1939] T 332a581a9a7d44c28f22b78e4b25baec P 8abc5118a51540278c04093762b87151: Deleting consensus metadata
I20260808 02:58:02.910430 11623 catalog_manager.cc:4841] TS 8abc5118a51540278c04093762b87151 (127.0.205.129:43055): tablet 332a581a9a7d44c28f22b78e4b25baec (table test-table [id=708d82ed24704ca6a02a80531914062c]) successfully deleted
Aug 08 02:58:02 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.129@KRBTEST.COM
I20260808 02:58:02.924355 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:02.896639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.0.205.129:43055)
0808 02:58:02.896837 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:02.896843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:02.906638 (+ 9795us) server_negotiation.cc:407] Connection header received
0808 02:58:02.906970 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:02.906977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:02.907062 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:02.907137 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:02.909010 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.910229 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.912122 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:02.912766 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:02.912896 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:02.918315 (+ 5419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:02.918332 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:02.918348 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:02.918385 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:02.921409 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:02.922368 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:02.922375 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:02.922377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:02.922508 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:02.922908 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:02.922911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:02.922913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:02.923289 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:02.923542 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:02.923849 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 02:58:02.924021 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:58:02.925067 11940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:03.056450 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 11751
I20260808 02:58:03.097369 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 11952
Aug 08 02:58:03 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.130@KRBTEST.COM
I20260808 02:58:03.114486 12491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:03.098272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34704 (local address 127.0.205.130:46157)
0808 02:58:03.098467 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:03.098475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:03.098494 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:58:03.098660 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:03.098664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:03.098734 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:03.098873 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:03.100193 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:03.100978 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:03.102665 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:03.103195 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:03.103339 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:03.108165 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:03.108182 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:03.108195 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:03.108236 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:03.111485 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:03.112198 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:03.112206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:03.112208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:03.112371 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:03.112795 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:03.112800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:03.112808 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:03.113201 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:03.113608 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:03.113844 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 02:58:03.114129 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:58:03.115272 12133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:03.243036 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 11952
I20260808 02:58:03.273954 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 12148
Aug 08 02:58:03 dist-test-slave-6ntk krb5kdc[11579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.205.131@KRBTEST.COM
I20260808 02:58:03.290973 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:03.274862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43314 (local address 127.0.205.131:35453)
0808 02:58:03.275325 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:03.275329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:03.275346 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:58:03.275406 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:03.275411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:03.275474 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:03.275575 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:03.276853 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:03.277599 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:03.279030 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:03.279423 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:03.279518 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:03.284083 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:03.284101 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:03.284118 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:03.284172 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:03.288046 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:03.288991 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:03.289000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:03.289002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:03.289151 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:03.289476 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:03.289479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:03.289481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:03.289849 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:03.290147 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:03.290388 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 02:58:03.290567 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":244,"threads_started":1}
I20260808 02:58:03.291733 12330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:03.455384 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 12148
I20260808 02:58:03.494472 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 11599
I20260808 02:58:03.495710 11704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:03.622041 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 11599
2026-08-08T02:58:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7390 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:58:03 dist-test-slave-6ntk krb5kdc[12515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:58:03 dist-test-slave-6ntk krb5kdc[12515](info): set up 2 sockets
Aug 08 02:58:03 dist-test-slave-6ntk krb5kdc[12515](info): commencing operation
krb5kdc: starting...
W20260808 02:58:04.988698 822 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:58:05 dist-test-slave-6ntk krb5kdc[12515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:58:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:58:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:58:05.291164 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:38781
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:38781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:58:05.649375 12533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:05.649763 12533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:05.650002 12533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:05.657811 12533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:58:05.658082 12533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:05.658156 12533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:05.658226 12533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:58:05.658277 12533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:58:05.668816 12533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:38781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:38781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:05.670336 12533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:05.671624 12533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:05.684655 12541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:05.684736 12543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:05.684756 12545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:05.684970 12533 server_base.cc:1034] running on GCE node
I20260808 02:58:05.686210 12533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:05.687704 12533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:05.688987 12533 hybrid_clock.cc:648] HybridClock initialized: now 1786157885688923 us; error 53 us; skew 500 ppm
I20260808 02:58:05.689366 12533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T02:58:05Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2136 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:58:05 dist-test-slave-6ntk krb5kdc[12551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:58:05 dist-test-slave-6ntk krb5kdc[12551](info): set up 2 sockets
Aug 08 02:58:05 dist-test-slave-6ntk krb5kdc[12551](info): commencing operation
krb5kdc: starting...
W20260808 02:58:07.098608 822 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:58:07 dist-test-slave-6ntk krb5kdc[12551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:58:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:58:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:58:07.391111 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43153
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:43153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:58:07.737145 12568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:07.737532 12568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:07.737715 12568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:07.745103 12568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:58:07.745276 12568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:07.745393 12568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:07.745515 12568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:58:07.745605 12568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:58:07.756057 12568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:43153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:07.757539 12568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:07.758949 12568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:07.771610 12576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:07.771804 12573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:07.771750 12574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:07.772807 12568 server_base.cc:1034] running on GCE node
I20260808 02:58:07.773674 12568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:07.775216 12568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:07.776449 12568 hybrid_clock.cc:648] HybridClock initialized: now 1786157887776394 us; error 37 us; skew 500 ppm
I20260808 02:58:07.776836 12568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:07 dist-test-slave-6ntk krb5kdc[12551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157887, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:07.780200 12568 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:58:07.781761 12568 webserver.cc:460] Webserver started at http://127.0.205.190:37773/ using document root <none> and password file <none>
I20260808 02:58:07.782541 12568 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:07.782716 12568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:07.783070 12568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:07.786389 12568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "d0f99ec0b1af484486357c0d6a0cc819"
format_stamp: "Formatted at 2026-08-08 02:58:07 on dist-test-slave-6ntk"
server_key: "680491d056d17ce40111b649cb335987"
server_key_iv: "7a4ef94cd2943fd8541cd405aa06cb51"
server_key_version: "encryptionkey@0"
I20260808 02:58:07.787247 12568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "d0f99ec0b1af484486357c0d6a0cc819"
format_stamp: "Formatted at 2026-08-08 02:58:07 on dist-test-slave-6ntk"
server_key: "680491d056d17ce40111b649cb335987"
server_key_iv: "7a4ef94cd2943fd8541cd405aa06cb51"
server_key_version: "encryptionkey@0"
I20260808 02:58:07.793263 12568 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260808 02:58:07.797196 12589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:07.798853 12568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:58:07.799098 12568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "d0f99ec0b1af484486357c0d6a0cc819"
format_stamp: "Formatted at 2026-08-08 02:58:07 on dist-test-slave-6ntk"
server_key: "680491d056d17ce40111b649cb335987"
server_key_iv: "7a4ef94cd2943fd8541cd405aa06cb51"
server_key_version: "encryptionkey@0"
I20260808 02:58:07.799340 12568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:07.817267 12568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:07.822021 12568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:07.822350 12568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:07.842495 12568 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:43153
I20260808 02:58:07.842602 12659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:43153 every 8 connection(s)
I20260808 02:58:07.844053 12568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:58:07.847970 12661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:07.850144 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 12568
I20260808 02:58:07.850368 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:58:07.851231 822 external_mini_cluster.cc:1428] Setting key 422ebbfa7cfb56ce2b3b9c63e11973ad
I20260808 02:58:07.861634 12661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819: Bootstrap starting.
I20260808 02:58:07.865825 12661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:07.867233 12661 log.cc:824] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:58:07 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:58:07.870880 12661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819: No bootstrap required, opened a new log
I20260808 02:58:07.876739 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:07.853598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.0.205.190:43153)
0808 02:58:07.854591 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:07.854603 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:07.854650 (+ 47us) server_negotiation.cc:407] Connection header received
0808 02:58:07.856353 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:07.856377 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:07.856789 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:07.857121 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:07.859131 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:07.860488 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:07.862347 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:07.863121 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:07.863277 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:07.868418 (+ 5141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:07.868436 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:07.868457 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:07.868519 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:07.871973 (+ 3454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:07.872964 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:07.872974 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:07.872981 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:07.873152 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:07.873585 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:07.873588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:07.873590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:07.874237 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:07.874516 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:07.875025 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 02:58:07.875532 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":500,"threads_started":1}
I20260808 02:58:07.880785 12661 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } }
I20260808 02:58:07.881203 12661 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:07.881325 12661 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f99ec0b1af484486357c0d6a0cc819, State: Initialized, Role: FOLLOWER
I20260808 02:58:07.881991 12661 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } }
I20260808 02:58:07.882270 12661 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:58:07.882491 12661 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:58:07.882685 12661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:07.885007 12661 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } }
I20260808 02:58:07.885640 12661 leader_election.cc:302] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0f99ec0b1af484486357c0d6a0cc819; no voters:
I20260808 02:58:07.886452 12661 leader_election.cc:288] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:58:07.886559 12671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:58:07.887383 12671 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 1 LEADER]: Becoming Leader. State: Replica: d0f99ec0b1af484486357c0d6a0cc819, State: Running, Role: LEADER
I20260808 02:58:07.888041 12671 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } }
I20260808 02:58:07.888748 12661 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:58:07.891070 12672 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0f99ec0b1af484486357c0d6a0cc819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } } }
I20260808 02:58:07.891587 12672 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [sys.catalog]: This master's current role is: LEADER
I20260808 02:58:07.891070 12673 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0f99ec0b1af484486357c0d6a0cc819. Latest consensus state: current_term: 1 leader_uuid: "d0f99ec0b1af484486357c0d6a0cc819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } } }
I20260808 02:58:07.891893 12673 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [sys.catalog]: This master's current role is: LEADER
I20260808 02:58:07.892692 12679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:58:07.899443 12679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:58:07.914777 12679 catalog_manager.cc:1347] Generated new cluster ID: 5ea62f7c61ea4983b6f9acdfda4421f0
I20260808 02:58:07.914928 12679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:58:07.927839 12679 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:58:07.929255 12679 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:58:07.939122 12679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819: Generated new TSK 0
I20260808 02:58:07.940029 12679 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:58:08.014639 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43153
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:58:08.374239 12699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:08.374547 12699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:08.374732 12699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:08.383086 12699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:08.383198 12699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:08.383411 12699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:58:08.394052 12699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:08.395551 12699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:08.396878 12699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:08.411358 12706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:08.411455 12710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:08.411360 12705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:08.412823 12699 server_base.cc:1034] running on GCE node
I20260808 02:58:08.413491 12699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:08.414572 12699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:08.415787 12699 hybrid_clock.cc:648] HybridClock initialized: now 1786157888415746 us; error 35 us; skew 500 ppm
I20260808 02:58:08.416198 12699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:08 dist-test-slave-6ntk krb5kdc[12551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157888, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:08.419667 12699 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:58:08.421267 12699 webserver.cc:460] Webserver started at http://127.0.205.129:38685/ using document root <none> and password file <none>
I20260808 02:58:08.422184 12699 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:08.422333 12699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:08.422693 12699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:08.425804 12699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "117c7bdbb8b841058791f98f4f15ed52"
format_stamp: "Formatted at 2026-08-08 02:58:08 on dist-test-slave-6ntk"
server_key: "445257daa85432bd963e0bb14d4187ae"
server_key_iv: "59f6d4797ef96ca1ecdfbb8da1893ed6"
server_key_version: "encryptionkey@0"
I20260808 02:58:08.426776 12699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "117c7bdbb8b841058791f98f4f15ed52"
format_stamp: "Formatted at 2026-08-08 02:58:08 on dist-test-slave-6ntk"
server_key: "445257daa85432bd963e0bb14d4187ae"
server_key_iv: "59f6d4797ef96ca1ecdfbb8da1893ed6"
server_key_version: "encryptionkey@0"
I20260808 02:58:08.432355 12699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:58:08.435977 12721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:08.437459 12699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:58:08.437640 12699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "117c7bdbb8b841058791f98f4f15ed52"
format_stamp: "Formatted at 2026-08-08 02:58:08 on dist-test-slave-6ntk"
server_key: "445257daa85432bd963e0bb14d4187ae"
server_key_iv: "59f6d4797ef96ca1ecdfbb8da1893ed6"
server_key_version: "encryptionkey@0"
I20260808 02:58:08.437850 12699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:08.457355 12699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:08.461565 12699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:08.461932 12699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:08.463047 12699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:08.464488 12699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:08.464568 12699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:08.464707 12699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:08.464771 12699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:08.499854 12699 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:39581
I20260808 02:58:08.499907 12892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:39581 every 8 connection(s)
I20260808 02:58:08.501267 12699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:58:08.503463 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 12699
I20260808 02:58:08.503683 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:58:08.504521 822 external_mini_cluster.cc:1428] Setting key 6e787df0827e1897bc14219b676bad84
Aug 08 02:58:08 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157888, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:58:08.524013 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:08.504291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:47105 (local address 127.0.205.190:43153)
0808 02:58:08.504829 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:08.504837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:08.506047 (+ 1210us) server_negotiation.cc:407] Connection header received
0808 02:58:08.507942 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:08.507948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:08.508052 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:08.508179 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:08.509733 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:08.510543 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:08.512133 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:08.512549 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:08.512701 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:08.516444 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:08.516454 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:08.516461 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:08.516506 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:08.520437 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:08.521190 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:08.521198 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:08.521204 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:08.521363 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:08.521811 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:08.521824 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:08.521831 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:08.522263 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:08.522443 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:08.523222 (+ 779us) server_negotiation.cc:299] Negotiation successful
0808 02:58:08.523512 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":259,"threads_started":1}
I20260808 02:58:08.524796 12897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:08.504978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:47105)
0808 02:58:08.505817 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:08.505926 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:08.507661 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:08.508439 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:08.508448 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:08.508980 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:08.509546 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:08.509564 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:08.510713 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:08.510718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:08.511970 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:08.511983 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:08.512807 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:08.512816 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:08.513066 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:08.513788 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:08.513816 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:08.516070 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:08.520585 (+ 4515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:08.520593 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:08.520606 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:08.521048 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:08.521495 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:08.521498 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:08.521500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:08.521688 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:08.522436 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:08.522448 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:08.522786 (+ 338us) client_negotiation.cc:769] Sending connection context
0808 02:58:08.523057 (+ 271us) client_negotiation.cc:240] Negotiation successful
0808 02:58:08.523499 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":357,"threads_started":1}
I20260808 02:58:08.526723 12893 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43153
I20260808 02:58:08.527086 12893 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:08.528105 12893 heartbeater.cc:511] Master 127.0.205.190:43153 requested a full tablet report, sending...
I20260808 02:58:08.530941 12614 ts_manager.cc:194] Registered new tserver with Master: 117c7bdbb8b841058791f98f4f15ed52 (127.0.205.129:39581)
I20260808 02:58:08.532631 12614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:47105
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:58:08.570600 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43153
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:58:08.925446 12903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:08.925956 12903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:08.926160 12903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:08.933748 12903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:08.933979 12903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:08.934182 12903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:58:08.944862 12903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:08.946424 12903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:08.947789 12903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:08.962498 12914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:08.962532 12911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:08.962760 12909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:08.964190 12903 server_base.cc:1034] running on GCE node
I20260808 02:58:08.964882 12903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:08.965976 12903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:08.967227 12903 hybrid_clock.cc:648] HybridClock initialized: now 1786157888967183 us; error 53 us; skew 500 ppm
I20260808 02:58:08.967582 12903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:08 dist-test-slave-6ntk krb5kdc[12551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157888, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:08.970865 12903 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:58:08.972435 12903 webserver.cc:460] Webserver started at http://127.0.205.130:45001/ using document root <none> and password file <none>
I20260808 02:58:08.973169 12903 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:08.973335 12903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:08.973685 12903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:08.976778 12903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "73d817615fdb4cd1b92310e21f1b26dc"
format_stamp: "Formatted at 2026-08-08 02:58:08 on dist-test-slave-6ntk"
server_key: "3bfdc54dcae358c114961e47c2881729"
server_key_iv: "5a0891d70aca98cb24cc4b813d248052"
server_key_version: "encryptionkey@0"
I20260808 02:58:08.977631 12903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "73d817615fdb4cd1b92310e21f1b26dc"
format_stamp: "Formatted at 2026-08-08 02:58:08 on dist-test-slave-6ntk"
server_key: "3bfdc54dcae358c114961e47c2881729"
server_key_iv: "5a0891d70aca98cb24cc4b813d248052"
server_key_version: "encryptionkey@0"
I20260808 02:58:08.983404 12903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 02:58:08.987150 12924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:08.988605 12903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 02:58:08.988790 12903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "73d817615fdb4cd1b92310e21f1b26dc"
format_stamp: "Formatted at 2026-08-08 02:58:08 on dist-test-slave-6ntk"
server_key: "3bfdc54dcae358c114961e47c2881729"
server_key_iv: "5a0891d70aca98cb24cc4b813d248052"
server_key_version: "encryptionkey@0"
I20260808 02:58:08.989015 12903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:09.019824 12903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:09.024199 12903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:09.024544 12903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:09.025607 12903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:09.027084 12903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:09.027199 12903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:09.027331 12903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:09.027428 12903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:09.062033 12903 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:38765
I20260808 02:58:09.062054 13084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:38765 every 8 connection(s)
I20260808 02:58:09.063409 12903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:58:09.069200 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 12903
I20260808 02:58:09.069402 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:58:09.069985 822 external_mini_cluster.cc:1428] Setting key 11d7ef67e0c972eb3ebc346de8a23d03
Aug 08 02:58:09 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157888, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:58:09.086218 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:09.066164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:57187 (local address 127.0.205.190:43153)
0808 02:58:09.066630 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:09.066635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:09.067602 (+ 967us) server_negotiation.cc:407] Connection header received
0808 02:58:09.069728 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:09.069733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:09.069810 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:09.070007 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:09.071823 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.072850 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.075527 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.075977 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.076082 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:09.080077 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:09.080082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:09.080086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:09.080119 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:09.082931 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:09.083715 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:09.083726 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:09.083728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:09.083864 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:09.084356 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:09.084359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:09.084362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:09.084638 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:09.084808 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:09.085536 (+ 728us) server_negotiation.cc:299] Negotiation successful
0808 02:58:09.085771 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":243,"threads_started":1}
I20260808 02:58:09.087258 13090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:09.066753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:57187)
0808 02:58:09.067428 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:09.067468 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:09.069369 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:09.070262 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:09.070274 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:09.070988 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:09.071633 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:09.071652 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.073034 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.073039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:09.075355 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:09.075376 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.076228 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.076237 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:09.076597 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:09.077456 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:09.077480 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:09.079807 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:09.083100 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:09.083110 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:09.083129 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:09.083570 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:09.083992 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:09.083997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:09.084000 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:09.084208 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:09.084781 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:09.084790 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:09.085155 (+ 365us) client_negotiation.cc:769] Sending connection context
0808 02:58:09.085443 (+ 288us) client_negotiation.cc:240] Negotiation successful
0808 02:58:09.086068 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":275,"threads_started":1}
I20260808 02:58:09.088970 13086 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43153
I20260808 02:58:09.089325 13086 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:09.090416 13086 heartbeater.cc:511] Master 127.0.205.190:43153 requested a full tablet report, sending...
I20260808 02:58:09.092239 12614 ts_manager.cc:194] Registered new tserver with Master: 73d817615fdb4cd1b92310e21f1b26dc (127.0.205.130:38765)
I20260808 02:58:09.094270 12614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:57187
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:58:09.137466 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43153
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:58:09.491700 13099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:09.491988 13099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:09.492228 13099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:09.500059 13099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:09.500170 13099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:09.500453 13099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:58:09.511020 13099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:09.512531 13099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:09.513991 13099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:09.528514 13107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:09.528513 13110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:09.528738 13106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:09.529309 13099 server_base.cc:1034] running on GCE node
I20260808 02:58:09.530339 13099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:09.531505 13099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:09.532794 13099 hybrid_clock.cc:648] HybridClock initialized: now 1786157889532717 us; error 54 us; skew 500 ppm
I20260808 02:58:09.533138 13099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:09 dist-test-slave-6ntk krb5kdc[12551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:09.536187 12893 heartbeater.cc:503] Master 127.0.205.190:43153 was elected leader, sending a full tablet report...
I20260808 02:58:09.536568 13099 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:58:09.538461 13099 webserver.cc:460] Webserver started at http://127.0.205.131:34995/ using document root <none> and password file <none>
I20260808 02:58:09.539202 13099 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:09.539359 13099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:09.539685 13099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:09.542882 13099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "f936c3dc87bf4f479eb668c636635e08"
format_stamp: "Formatted at 2026-08-08 02:58:09 on dist-test-slave-6ntk"
server_key: "67a9cb18250227de05ad50c36bf329be"
server_key_iv: "e6b61f83d05ca518aae62cc91423c380"
server_key_version: "encryptionkey@0"
I20260808 02:58:09.543718 13099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "f936c3dc87bf4f479eb668c636635e08"
format_stamp: "Formatted at 2026-08-08 02:58:09 on dist-test-slave-6ntk"
server_key: "67a9cb18250227de05ad50c36bf329be"
server_key_iv: "e6b61f83d05ca518aae62cc91423c380"
server_key_version: "encryptionkey@0"
I20260808 02:58:09.549141 13099 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260808 02:58:09.552845 13121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:09.554287 13099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 02:58:09.554469 13099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "f936c3dc87bf4f479eb668c636635e08"
format_stamp: "Formatted at 2026-08-08 02:58:09 on dist-test-slave-6ntk"
server_key: "67a9cb18250227de05ad50c36bf329be"
server_key_iv: "e6b61f83d05ca518aae62cc91423c380"
server_key_version: "encryptionkey@0"
I20260808 02:58:09.554688 13099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:09.571880 13099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:09.576210 13099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:09.576534 13099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:09.577538 13099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:09.579097 13099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:09.579219 13099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:09.579383 13099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:09.579466 13099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:09.614825 13099 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:42301
I20260808 02:58:09.614871 13293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:42301 every 8 connection(s)
I20260808 02:58:09.616273 13099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:58:09.626694 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 13099
I20260808 02:58:09.626928 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:58:09.627650 822 external_mini_cluster.cc:1428] Setting key 4d83e1320f280df42f877ae941d90394
Aug 08 02:58:09 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:58:09.638911 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:09.619340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:37569 (local address 127.0.205.190:43153)
0808 02:58:09.619867 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:09.619874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:09.620860 (+ 986us) server_negotiation.cc:407] Connection header received
0808 02:58:09.622802 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:09.622806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:09.622884 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:09.622993 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:09.624587 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.625431 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.627333 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.628072 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.628241 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:09.632247 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:09.632252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:09.632256 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:09.632293 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:09.635071 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:09.635944 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:09.635951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:09.635956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:09.636122 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:09.636668 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:09.636674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:09.636679 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:09.637074 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:09.637224 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:09.638133 (+ 909us) server_negotiation.cc:299] Negotiation successful
0808 02:58:09.638417 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":237,"threads_started":1}
I20260808 02:58:09.639783 13297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:09.619931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:37569)
0808 02:58:09.620674 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:09.620722 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:09.622480 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:09.623234 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:09.623243 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:09.623776 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:09.624393 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:09.624411 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.625595 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.625599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:09.627148 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:09.627167 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.628351 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.628360 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:09.628685 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:09.629483 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:09.629508 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:09.632027 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:09.635228 (+ 3201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:09.635237 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:09.635252 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:09.635770 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:09.636296 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:09.636300 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:09.636303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:09.636532 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:09.637229 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:09.637239 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:09.637628 (+ 389us) client_negotiation.cc:769] Sending connection context
0808 02:58:09.638045 (+ 417us) client_negotiation.cc:240] Negotiation successful
0808 02:58:09.638503 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":302,"threads_started":1}
I20260808 02:58:09.641463 13294 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43153
I20260808 02:58:09.641929 13294 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:09.642998 13294 heartbeater.cc:511] Master 127.0.205.190:43153 requested a full tablet report, sending...
I20260808 02:58:09.644779 12614 ts_manager.cc:194] Registered new tserver with Master: f936c3dc87bf4f479eb668c636635e08 (127.0.205.131:42301)
I20260808 02:58:09.645581 12614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:37569
I20260808 02:58:09.656705 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab.
Entry for principal oryx/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab.
WARNING: no policy specified for oryx/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab.
Entry for principal oryx/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab.
WARNING: no policy specified for oryx/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab.
Entry for principal oryx/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab.
I20260808 02:58:09.854405 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 12699
Aug 08 02:58:09 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:58:09.875027 13308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:09.855780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46688 (local address 127.0.205.129:39581)
0808 02:58:09.856523 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:09.856531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:09.856559 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:58:09.856624 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:09.856630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:09.856865 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:09.857002 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:09.859098 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.860232 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.862400 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.862991 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.863154 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:09.867371 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:09.867383 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:09.867404 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:09.867439 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:09.870514 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:09.871271 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:09.871281 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:09.871284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:09.871458 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:09.871963 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:09.871968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:09.871974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:09.872427 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:09.872757 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:09.874124 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0808 02:58:09.874437 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":351,"threads_started":1}
I20260808 02:58:09.876883 12883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:09.964359 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 12699
I20260808 02:58:09.989266 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 12903
Aug 08 02:58:09 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:58:10.006742 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:09.990356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45570 (local address 127.0.205.130:38765)
0808 02:58:09.990962 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:09.990969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:09.990995 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:58:09.991063 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:09.991069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:09.991275 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:09.991425 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:09.992773 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.994197 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.995529 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:09.996073 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:09.996177 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:10.000782 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:10.000794 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:10.000809 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:10.000842 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:10.004062 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:10.004706 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:10.004712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:10.004714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:10.004843 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:10.005180 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:10.005184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:10.005185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:10.005530 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:10.005754 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:10.006090 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 02:58:10.006295 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":323,"threads_started":1}
I20260808 02:58:10.008121 13077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:10.094149 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 12903
I20260808 02:58:10.119959 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 13099
Aug 08 02:58:10 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:58:10.137225 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:10.121040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44990 (local address 127.0.205.131:42301)
0808 02:58:10.121611 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:10.121619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:10.121649 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:58:10.121714 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:10.121720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:10.122047 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:10.122181 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:10.123677 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:10.124691 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:10.126158 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:10.126721 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:10.126831 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:10.131371 (+ 4540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:10.131384 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:10.131411 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:10.131448 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:10.134683 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:10.135298 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:10.135305 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:10.135307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:10.135438 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:10.135771 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:10.135774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:10.135776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:10.136119 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:10.136357 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:10.136600 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 02:58:10.136841 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":259,"threads_started":1}
I20260808 02:58:10.138695 13285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:10.229395 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 13099
I20260808 02:58:10.255858 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 12568
I20260808 02:58:10.257181 12654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:10.366006 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 12568
I20260808 02:58:10.394770 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43153
--webserver_interface=127.0.205.190
--webserver_port=37773
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:43153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:58:10.744261 13317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:10.744594 13317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:10.744686 13317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:10.751971 13317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:58:10.752099 13317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:10.752147 13317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:10.752214 13317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:58:10.752260 13317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:58:10.762326 13317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:43153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=37773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:10.763653 13317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:10.765004 13317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:10.777328 13325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:10.777328 13326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:10.777555 13328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:10.778431 13317 server_base.cc:1034] running on GCE node
I20260808 02:58:10.779260 13317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:10.780817 13317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:10.782073 13317 hybrid_clock.cc:648] HybridClock initialized: now 1786157890782021 us; error 64 us; skew 500 ppm
I20260808 02:58:10.782452 13317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:10 dist-test-slave-6ntk krb5kdc[12551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157890, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:10.790951 13317 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:58:10.792860 13317 webserver.cc:460] Webserver started at http://127.0.205.190:37773/ using document root <none> and password file <none>
I20260808 02:58:10.793654 13317 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:10.793834 13317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:10.799717 13317 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 02:58:10.802747 13339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:10.804128 13317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:58:10.804335 13317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "d0f99ec0b1af484486357c0d6a0cc819"
format_stamp: "Formatted at 2026-08-08 02:58:07 on dist-test-slave-6ntk"
server_key: "680491d056d17ce40111b649cb335987"
server_key_iv: "7a4ef94cd2943fd8541cd405aa06cb51"
server_key_version: "encryptionkey@0"
I20260808 02:58:10.805500 13317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:10.817272 13317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:10.821462 13317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:10.821730 13317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:10.841404 13317 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:43153
I20260808 02:58:10.841413 13409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:43153 every 8 connection(s)
I20260808 02:58:10.842957 13317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:58:10.844043 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 13317
I20260808 02:58:10.844967 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:39581
--local_ip_for_outbound_sockets=127.0.205.129
--tserver_master_addrs=127.0.205.190:43153
--webserver_port=38685
--webserver_interface=127.0.205.129
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.205.129
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:58:10.850315 13410 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:58:10.854130 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819: Bootstrap starting.
I20260808 02:58:10.873308 13410 log.cc:824] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819: Log is configured to *not* fsync() on all Append() calls
I20260808 02:58:10.882633 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:58:10.883200 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819: Bootstrap complete.
I20260808 02:58:10.892572 13410 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } }
I20260808 02:58:10.893435 13410 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f99ec0b1af484486357c0d6a0cc819, State: Initialized, Role: FOLLOWER
I20260808 02:58:10.894227 13410 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } }
I20260808 02:58:10.894513 13410 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:58:10.894691 13410 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:58:10.894908 13410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:58:10.898988 13410 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } }
I20260808 02:58:10.899537 13410 leader_election.cc:302] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0f99ec0b1af484486357c0d6a0cc819; no voters:
I20260808 02:58:10.900265 13410 leader_election.cc:288] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:58:10.900454 13417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:58:10.901897 13417 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [term 2 LEADER]: Becoming Leader. State: Replica: d0f99ec0b1af484486357c0d6a0cc819, State: Running, Role: LEADER
I20260808 02:58:10.902534 13417 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } }
I20260808 02:58:10.902894 13410 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:58:10.905205 13419 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0f99ec0b1af484486357c0d6a0cc819. Latest consensus state: current_term: 2 leader_uuid: "d0f99ec0b1af484486357c0d6a0cc819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } } }
I20260808 02:58:10.905627 13419 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [sys.catalog]: This master's current role is: LEADER
I20260808 02:58:10.906360 13418 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d0f99ec0b1af484486357c0d6a0cc819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f99ec0b1af484486357c0d6a0cc819" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43153 } } }
I20260808 02:58:10.906700 13418 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819 [sys.catalog]: This master's current role is: LEADER
I20260808 02:58:10.916190 13430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:58:10.922324 13430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:58:10.924177 13430 catalog_manager.cc:1259] Loaded cluster ID: 5ea62f7c61ea4983b6f9acdfda4421f0
I20260808 02:58:10.924377 13430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:58:10.929069 13440 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 02:58:10.929524 13430 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:58:10.931159 13430 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0f99ec0b1af484486357c0d6a0cc819: Loaded TSK: 0
I20260808 02:58:10.932384 13430 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:58:11.247918 13413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:11.248246 13413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:11.248409 13413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:11.255960 13413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:11.256079 13413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:11.256268 13413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:58:11.266713 13413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.129.keytab
--principal=oryx/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:39581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=38685
--tserver_master_addrs=127.0.205.190:43153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:11.268208 13413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:11.269639 13413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:11.284161 13447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:11.284916 13449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:11.284947 13446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:11.285146 13413 server_base.cc:1034] running on GCE node
I20260808 02:58:11.286291 13413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:11.287561 13413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:11.288832 13413 hybrid_clock.cc:648] HybridClock initialized: now 1786157891288773 us; error 44 us; skew 500 ppm
I20260808 02:58:11.289227 13413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:11 dist-test-slave-6ntk krb5kdc[12551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157891, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:11.293363 13413 init.cc:377] Logged in from keytab as oryx/127.0.205.129@KRBTEST.COM (short username oryx)
I20260808 02:58:11.295192 13413 webserver.cc:460] Webserver started at http://127.0.205.129:38685/ using document root <none> and password file <none>
I20260808 02:58:11.296073 13413 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:11.296222 13413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:11.303586 13413 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:58:11.307092 13460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:11.308884 13413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:58:11.309067 13413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "117c7bdbb8b841058791f98f4f15ed52"
format_stamp: "Formatted at 2026-08-08 02:58:08 on dist-test-slave-6ntk"
server_key: "445257daa85432bd963e0bb14d4187ae"
server_key_iv: "59f6d4797ef96ca1ecdfbb8da1893ed6"
server_key_version: "encryptionkey@0"
I20260808 02:58:11.310552 13413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:11.331519 13413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:11.337095 13413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:11.337395 13413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:11.338765 13413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:11.340476 13413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:11.340591 13413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:11.340694 13413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:11.340770 13413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:11.377691 13413 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:39581
I20260808 02:58:11.377794 13613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:39581 every 8 connection(s)
I20260808 02:58:11.379258 13413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:58:11.386530 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 13413
I20260808 02:58:11.387341 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:38765
--local_ip_for_outbound_sockets=127.0.205.130
--tserver_master_addrs=127.0.205.190:43153
--webserver_port=45001
--webserver_interface=127.0.205.130
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.205.130
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:58:11 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:11 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:58:11.788465 13619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:11.788764 13619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:11.788918 13619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:11.796679 13619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:11.796804 13619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:11.797000 13619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:58:11.807835 13619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.130.keytab
--principal=oryx/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:38765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=45001
--tserver_master_addrs=127.0.205.190:43153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:11.809425 13619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:11.810925 13619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:11.830109 13626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:11.830109 13628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:11.831385 13631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:11.832289 13619 server_base.cc:1034] running on GCE node
I20260808 02:58:11.833024 13619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:11.834357 13619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:11.835609 13619 hybrid_clock.cc:648] HybridClock initialized: now 1786157891835553 us; error 47 us; skew 500 ppm
I20260808 02:58:11.835976 13619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:11 dist-test-slave-6ntk krb5kdc[12551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157891, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:11.839998 13619 init.cc:377] Logged in from keytab as oryx/127.0.205.130@KRBTEST.COM (short username oryx)
I20260808 02:58:11.842341 13619 webserver.cc:460] Webserver started at http://127.0.205.130:45001/ using document root <none> and password file <none>
I20260808 02:58:11.843235 13619 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:11.843410 13619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:11.851179 13619 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 02:58:11.855103 13641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:11.857326 13619 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 02:58:11.857581 13619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "73d817615fdb4cd1b92310e21f1b26dc"
format_stamp: "Formatted at 2026-08-08 02:58:08 on dist-test-slave-6ntk"
server_key: "3bfdc54dcae358c114961e47c2881729"
server_key_iv: "5a0891d70aca98cb24cc4b813d248052"
server_key_version: "encryptionkey@0"
I20260808 02:58:11.859598 13619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:11.916189 13619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:11.921500 13619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:11.921947 13619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:11.923125 13619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:11.924618 13619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:11.924770 13619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:11.924916 13619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:11.925010 13619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:11.968617 13619 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:38765
I20260808 02:58:11.968679 13810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:38765 every 8 connection(s)
I20260808 02:58:11.970293 13619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:58:11.976715 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 13619
I20260808 02:58:11.977674 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:42301
--local_ip_for_outbound_sockets=127.0.205.131
--tserver_master_addrs=127.0.205.190:43153
--webserver_port=34995
--webserver_interface=127.0.205.131
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:44135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.205.131
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:58:11 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:11 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:58:12.671165 13817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:12.671489 13817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:12.671672 13817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:12.680032 13817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:12.680226 13817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:12.680464 13817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:58:12.691938 13817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:44135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/krb5kdc/oryx_127.0.205.131.keytab
--principal=oryx/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:42301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=34995
--tserver_master_addrs=127.0.205.190:43153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:12.693461 13817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:12.694988 13817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:12.716248 13824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:12.720419 13825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:12.723285 13828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:12.723793 13817 server_base.cc:1034] running on GCE node
I20260808 02:58:12.725095 13817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:12.726936 13817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:12.728354 13817 hybrid_clock.cc:648] HybridClock initialized: now 1786157892727969 us; error 302 us; skew 500 ppm
I20260808 02:58:12.728961 13817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:12 dist-test-slave-6ntk krb5kdc[12551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157892, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:12.735082 13817 init.cc:377] Logged in from keytab as oryx/127.0.205.131@KRBTEST.COM (short username oryx)
I20260808 02:58:12.737603 13817 webserver.cc:460] Webserver started at http://127.0.205.131:34995/ using document root <none> and password file <none>
I20260808 02:58:12.738651 13817 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:12.738819 13817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:12.747121 13817 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.003s
I20260808 02:58:12.751370 13836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:12.753408 13817 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260808 02:58:12.753599 13817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "f936c3dc87bf4f479eb668c636635e08"
format_stamp: "Formatted at 2026-08-08 02:58:09 on dist-test-slave-6ntk"
server_key: "67a9cb18250227de05ad50c36bf329be"
server_key_iv: "e6b61f83d05ca518aae62cc91423c380"
server_key_version: "encryptionkey@0"
I20260808 02:58:12.755223 13817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:12.785519 13817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:12.791550 13817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:12.791850 13817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:12.793121 13817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:12.795025 13817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:12.795137 13817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:12.795284 13817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:12.795360 13817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260808 02:58:12.819588 13618 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.425s user 0.000s sys 0.016s
I20260808 02:58:12.822281 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:11.382522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:51347)
0808 02:58:11.383365 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:11.383412 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:11.385135 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:11.386726 (+ 1591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:11.386736 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:11.387551 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:11.388389 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:11.388419 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:11.390184 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:11.390192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:11.392336 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:11.392356 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:11.393425 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:11.393436 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:11.393758 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:11.394670 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:11.394705 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:12.820710 (+1426005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":366,"threads_started":1}
W20260808 02:58:12.822939 13617 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:11.382442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:51347 (local address 127.0.205.190:43153)
0808 02:58:11.383173 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:11.383184 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:11.383551 (+ 367us) server_negotiation.cc:407] Connection header received
0808 02:58:11.385695 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:11.385708 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:11.386166 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:11.386470 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:11.388631 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:11.390007 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:11.392526 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:11.393168 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:11.393312 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:12.821593 (+1428281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:51347: BlockingRecv error: recv got EOF from 127.0.205.129:51347 (error 108)
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":277,"threads_started":1}
W20260808 02:58:12.824174 13615 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:43153 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.205.190:43153: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Aug 08 02:58:12 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:12 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:12.847409 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:12.825298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:48681)
0808 02:58:12.825548 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:12.825575 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:12.825736 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:12.826871 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:12.826880 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:12.827587 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:12.827964 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:12.827982 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.830319 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.830328 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:12.832678 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:12.832703 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.834100 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.834110 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:12.834398 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:12.835375 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:12.835405 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:12.847009 (+ 11604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 02:58:12.848361 13617 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:12.825896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:48681 (local address 127.0.205.190:43153)
0808 02:58:12.826152 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:12.826161 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:12.826189 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:58:12.826276 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:12.826284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:12.826398 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:12.826541 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:12.828227 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.830050 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.832944 (+ 2894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.833787 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.834867 (+ 1080us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:12.848042 (+ 13175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:48681: BlockingRecv error: recv got EOF from 127.0.205.129:48681 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 02:58:12 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:12 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:12.865770 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:12.848695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:51179)
0808 02:58:12.849922 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:12.849949 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:12.850110 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:12.850827 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:12.850835 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:12.851457 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:12.851773 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:12.851791 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.854079 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.854088 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:12.856334 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:12.856357 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.858099 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.858109 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:12.858391 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:12.859364 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:12.859401 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:12.865543 (+ 6142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1042}
W20260808 02:58:12.866369 13617 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:12.849233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:51179 (local address 127.0.205.190:43153)
0808 02:58:12.849688 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:12.849698 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:12.850249 (+ 551us) server_negotiation.cc:407] Connection header received
0808 02:58:12.850336 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:12.850344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:12.850450 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:12.850572 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:12.852013 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.853687 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.856588 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.857543 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.857711 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:12.865979 (+ 8268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:51179: BlockingRecv error: recv got EOF from 127.0.205.129:51179 (error 108)
Metrics: {"server-negotiator.queue_time_us":285}
W20260808 02:58:12.866824 13615 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:58:12.874976 13817 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:42301
I20260808 02:58:12.875056 13998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:42301 every 8 connection(s)
I20260808 02:58:12.876462 13817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:58:12.884768 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 13817
Aug 08 02:58:12 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:12 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:12.906242 14004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:12.885953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.0.205.190:43153)
0808 02:58:12.887057 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:12.887062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:12.887080 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:58:12.887127 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:12.887131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:12.887209 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:12.889447 (+ 2238us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:12.889699 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.890771 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.892489 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.893524 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.893720 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:12.898174 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:12.898188 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:12.898210 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:12.898259 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:12.901762 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:12.902523 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:12.902533 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:12.902544 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:12.902796 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:12.903872 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:12.903878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:12.903882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:12.904473 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:12.904804 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:12.905272 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 02:58:12.905814 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":985,"thread_start_us":813,"threads_started":1}
W20260808 02:58:13.133108 13816 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.138s user 0.001s sys 0.014s
W20260808 02:58:13.134711 13815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:11.973339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:47291 (local address 127.0.205.190:43153)
0808 02:58:11.982488 (+ 9149us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:11.982500 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:11.982538 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:58:11.989175 (+ 6637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:11.989180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:11.989266 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:11.989404 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:11.991239 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:11.992039 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:11.993638 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:11.994148 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:11.994303 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:13.134343 (+1140040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:47291: BlockingRecv error: recv got EOF from 127.0.205.130:47291 (error 108)
Metrics: {"server-negotiator.queue_time_us":8939,"thread_start_us":8784,"threads_started":1}
I20260808 02:58:13.135313 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:11.974528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:47291)
0808 02:58:11.975740 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:11.975806 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:11.988621 (+ 12815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:11.989725 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:11.989735 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:11.990383 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:11.991002 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:11.991022 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:11.992222 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:11.992231 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:11.993478 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:11.993491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:11.994431 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:11.994439 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:11.994691 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:11.995440 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:11.995464 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:13.134096 (+1138632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":465,"threads_started":1}
W20260808 02:58:13.136425 13812 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:43153 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.205.190:43153: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Aug 08 02:58:13 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:13 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:13.147744 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:13.137338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:46141)
0808 02:58:13.137598 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:13.137614 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:13.137774 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:13.138199 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:13.138204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:13.138618 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:13.138791 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:13.138799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.139896 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.139902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:13.140876 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:13.140888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.141655 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.141662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:13.141843 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:13.142775 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:13.142793 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:13.147430 (+ 4637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116}
W20260808 02:58:13.147877 13815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:13.137333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:46141 (local address 127.0.205.190:43153)
0808 02:58:13.137528 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:13.137535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:13.137697 (+ 162us) server_negotiation.cc:407] Connection header received
0808 02:58:13.137983 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:13.137988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:13.138054 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:13.138168 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:13.138967 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.139725 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.141033 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.141464 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.141613 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:13.147635 (+ 6022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:46141: BlockingRecv error: recv got EOF from 127.0.205.130:46141 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 02:58:13 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:13 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:13.161572 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:13.149024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:46133)
0808 02:58:13.149373 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:13.149396 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:13.149555 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:13.149994 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:13.150002 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:13.150610 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:13.150945 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:13.150962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.152041 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.152050 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:13.154603 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:13.154629 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.155453 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.155463 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:13.155784 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:13.156670 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:13.156707 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:13.161254 (+ 4547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182}
W20260808 02:58:13.161772 13815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:13.149231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:46133 (local address 127.0.205.190:43153)
0808 02:58:13.149409 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:13.149413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:13.149462 (+ 49us) server_negotiation.cc:407] Connection header received
0808 02:58:13.149690 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:13.149693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:13.149751 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:13.149829 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:13.151122 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.151864 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.154794 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.155241 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.155348 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:13.161471 (+ 6123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:46133: BlockingRecv error: recv got EOF from 127.0.205.130:46133 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 02:58:13.162309 13812 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 02:58:13 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:13 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:13.882927 14007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:13.867584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:56689)
0808 02:58:13.868427 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:13.868459 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:13.868694 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:13.869162 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:13.869168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:13.870069 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:13.870364 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:13.870382 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.871561 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.871571 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:13.873960 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:13.873983 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.875077 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.875088 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:13.875390 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:13.876371 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:13.876407 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:13.882464 (+ 6057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":459,"threads_started":1}
W20260808 02:58:13.883261 14009 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:13.867661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:56689 (local address 127.0.205.190:43153)
0808 02:58:13.868696 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:13.868704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:13.868725 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:58:13.868863 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:13.868869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:13.868964 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:13.869060 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:13.870563 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.871374 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.874173 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:13.874790 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:13.874962 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:13.882881 (+ 7919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:56689: BlockingRecv error: recv got EOF from 127.0.205.129:56689 (error 108)
Metrics: {"server-negotiator.queue_time_us":924,"thread_start_us":809,"threads_started":1}
W20260808 02:58:14.024870 14001 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.129s user 0.000s sys 0.028s
W20260808 02:58:14.026249 13617 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:12.880449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:46563 (local address 127.0.205.190:43153)
0808 02:58:12.880745 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:12.880755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:12.882245 (+ 1490us) server_negotiation.cc:407] Connection header received
0808 02:58:12.886775 (+ 4530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:12.886783 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:12.886890 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:12.886973 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:12.889542 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.891293 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.893352 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.894040 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.894178 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:14.025975 (+1131797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:46563: BlockingRecv error: recv got EOF from 127.0.205.131:46563 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 02:58:14.026916 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:12.880883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:46563)
0808 02:58:12.881993 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:12.882101 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:12.886427 (+ 4326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:12.887559 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:12.887575 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:12.888375 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:12.889290 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:12.889318 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.891532 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.891539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:12.893175 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:12.893189 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:12.894375 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:12.894386 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:12.894857 (+ 471us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:12.895845 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:12.895902 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:14.025749 (+1129847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":360,"threads_started":1}
W20260808 02:58:14.027896 13999 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:43153 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.205.190:43153: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Aug 08 02:58:14 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:14 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:14.037820 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:14.028547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:36581)
0808 02:58:14.028809 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:14.028867 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:14.029028 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:14.029384 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:14.029389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:14.029828 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:14.030067 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:14.030077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.031261 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.031269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:14.032264 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:14.032277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.033085 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.033092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:14.033273 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:14.033948 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:14.033969 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:14.037593 (+ 3624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136}
W20260808 02:58:14.038188 13617 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:14.028736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:36581 (local address 127.0.205.190:43153)
0808 02:58:14.029011 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:14.029017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:14.029038 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:58:14.029171 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:14.029175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:14.029235 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:14.029351 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:14.030239 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.031019 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.032434 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.032889 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.033049 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:14.037826 (+ 4777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:36581: BlockingRecv error: recv got EOF from 127.0.205.131:36581 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
Aug 08 02:58:14 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:14 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:14.047278 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:14.038792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:54183)
0808 02:58:14.038999 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:14.039020 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:14.039137 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:14.039530 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:14.039534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:14.039863 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:14.040019 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:14.040028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.041060 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.041066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:14.042107 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:14.042120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.042858 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.042864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:14.043049 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:14.043620 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:14.043638 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:14.047075 (+ 3437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 02:58:14.047513 13617 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:14.038886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:54183 (local address 127.0.205.190:43153)
0808 02:58:14.039080 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:14.039087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:14.039104 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:58:14.039293 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:14.039297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:14.039372 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:14.039493 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:14.040170 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.040901 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.042254 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.042687 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.042845 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:14.047287 (+ 4442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:54183: BlockingRecv error: recv got EOF from 127.0.205.131:54183 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 02:58:14.047734 13999 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 02:58:14 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:14 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:14.173569 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:14.163007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:47277)
0808 02:58:14.163644 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:14.163661 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:14.163785 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:14.164187 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:14.164190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:14.164611 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:14.164761 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:14.164769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.165838 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.165841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:14.167214 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:14.167228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.168388 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.168395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:14.168627 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:14.169257 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:14.169275 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:14.173208 (+ 3933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":378,"threads_started":1}
W20260808 02:58:14.173733 13617 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:14.163138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:47277 (local address 127.0.205.190:43153)
0808 02:58:14.163394 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:14.163402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:14.163753 (+ 351us) server_negotiation.cc:407] Connection header received
0808 02:58:14.163933 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:14.163937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:14.164025 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:14.164119 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:14.164907 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.165709 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.167365 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.168127 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.168338 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:14.173461 (+ 5123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:47277: BlockingRecv error: recv got EOF from 127.0.205.130:47277 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
Aug 08 02:58:14 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:14 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:14.894424 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:14.884172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:42509)
0808 02:58:14.884793 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:14.884819 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:14.884970 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:14.885552 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:14.885558 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:14.886122 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:14.886314 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:14.886324 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.887448 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.887454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:14.888476 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:14.888489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.889261 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.889268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:14.889469 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:14.890181 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:14.890199 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:14.893958 (+ 3759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":347,"threads_started":1}
W20260808 02:58:14.894462 14012 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:14.884290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:42509 (local address 127.0.205.190:43153)
0808 02:58:14.885158 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:14.885193 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:14.885214 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:58:14.885274 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:14.885279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:14.885361 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:14.885508 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:14.886482 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.887285 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.888640 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:14.889067 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:14.889211 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:14.894189 (+ 4978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:42509: BlockingRecv error: recv got EOF from 127.0.205.129:42509 (error 108)
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":600,"threads_started":1}
Aug 08 02:58:15 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:15 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:15.058300 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:15.048461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:48013)
0808 02:58:15.049082 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:15.049099 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:15.049219 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:15.049702 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:15.049705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:15.050214 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:15.050369 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:15.050378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.051545 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.051550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:15.052604 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:15.052617 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.053347 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.053351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:15.053468 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:15.054203 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:15.054223 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:15.058050 (+ 3827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":376,"threads_started":1}
W20260808 02:58:15.058547 14012 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:15.048514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:48013 (local address 127.0.205.190:43153)
0808 02:58:15.048705 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:15.048710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:15.049184 (+ 474us) server_negotiation.cc:407] Connection header received
0808 02:58:15.049320 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:15.049323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:15.049388 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:15.049686 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:15.050523 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.051367 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.052767 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.053171 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.053280 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:15.058279 (+ 4999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:48013: BlockingRecv error: recv got EOF from 127.0.205.131:48013 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 02:58:15 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:15 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:15.184849 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:15.174647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:57315)
0808 02:58:15.175215 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:15.175231 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:15.175400 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:15.175807 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:15.175811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:15.176268 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:15.176455 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:15.176464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.177721 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.177726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:15.178748 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:15.178760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.179484 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.179490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:15.179673 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:15.180282 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:15.180301 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:15.184592 (+ 4291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":330,"threads_started":1}
W20260808 02:58:15.185014 14012 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:15.174733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:57315 (local address 127.0.205.190:43153)
0808 02:58:15.175015 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:15.175024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:15.175373 (+ 349us) server_negotiation.cc:407] Connection header received
0808 02:58:15.175543 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:15.175549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:15.175637 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:15.175767 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:15.176715 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.177550 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.178903 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.179316 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.179461 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:15.184775 (+ 5314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:57315: BlockingRecv error: recv got EOF from 127.0.205.130:57315 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 08 02:58:15 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:15 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:15.909757 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:15.895623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:35535)
0808 02:58:15.896130 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:15.896147 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:15.896262 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:15.896868 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:15.896873 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:15.897353 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:15.897632 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:15.897646 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.898963 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.898967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:15.899907 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:15.899919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.900635 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.900639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:15.900780 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:15.905403 (+ 4623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:15.905423 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:15.909343 (+ 3920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":284,"threads_started":1}
W20260808 02:58:15.909965 14018 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:15.895902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:35535 (local address 127.0.205.190:43153)
0808 02:58:15.896518 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:15.896523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:15.896542 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:58:15.896597 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:15.896607 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:15.896667 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:15.896801 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:15.897862 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.898787 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.900065 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:15.900500 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:15.900601 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:15.909613 (+ 9012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:35535: BlockingRecv error: recv got EOF from 127.0.205.129:35535 (error 108)
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":341,"threads_started":1}
Aug 08 02:58:16 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:16 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:16.069501 14019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:16.059401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:37817)
0808 02:58:16.059914 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:16.059931 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:16.060057 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:16.060364 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:16.060367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:16.060911 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:16.061103 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:16.061113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.062284 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.062288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:16.063226 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:16.063237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.063921 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.063925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:16.064048 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:16.064717 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:16.064740 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:16.069079 (+ 4339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":279,"threads_started":1}
W20260808 02:58:16.069590 14018 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:16.059563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:37817 (local address 127.0.205.190:43153)
0808 02:58:16.059797 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:16.059804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:16.060030 (+ 226us) server_negotiation.cc:407] Connection header received
0808 02:58:16.060166 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:16.060170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:16.060234 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:16.060316 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:16.061238 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.062136 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.063366 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.063771 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.063872 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:16.069322 (+ 5450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:37817: BlockingRecv error: recv got EOF from 127.0.205.131:37817 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 08 02:58:16 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:16 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:16.199981 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:16.186762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:39571)
0808 02:58:16.188228 (+ 1466us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:16.188249 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:16.188431 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:16.188936 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:16.188942 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:16.189514 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:16.189777 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:16.189794 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.191397 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.191402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:16.192865 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:16.192883 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.193993 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.194000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:16.194215 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:16.194946 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:16.194973 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:16.199570 (+ 4597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1345,"thread_start_us":1197,"threads_started":1}
W20260808 02:58:16.200073 14018 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:16.187570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:39571 (local address 127.0.205.190:43153)
0808 02:58:16.187785 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:16.187794 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:16.188357 (+ 563us) server_negotiation.cc:407] Connection header received
0808 02:58:16.188611 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:16.188620 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:16.188739 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:16.188867 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:16.189977 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.191172 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.193011 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.193714 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.193855 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:16.199794 (+ 5939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:39571: BlockingRecv error: recv got EOF from 127.0.205.130:39571 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 02:58:16 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:16 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:16.920504 14022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:16.910770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:53119)
0808 02:58:16.911336 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:16.911353 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:16.911475 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:16.911851 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:16.911854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:16.912282 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:16.912467 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:16.912476 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.913612 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.913618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:16.914754 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:16.914767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.915487 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.915491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:16.915616 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:16.916274 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:16.916291 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:16.920083 (+ 3792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":296,"threads_started":1}
W20260808 02:58:16.920616 14028 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:16.910880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:53119 (local address 127.0.205.190:43153)
0808 02:58:16.911452 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:16.911457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:16.911476 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:58:16.911579 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:16.911583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:16.911661 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:16.911775 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:16.912630 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.913468 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.914887 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:16.915287 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:16.915457 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:16.920322 (+ 4865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:53119: BlockingRecv error: recv got EOF from 127.0.205.129:53119 (error 108)
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":296,"threads_started":1}
Aug 08 02:58:17 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:17 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:17.079226 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:17.070480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:48113)
0808 02:58:17.070974 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:17.070991 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:17.071109 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:17.071430 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:17.071433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:17.071797 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:17.071982 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:17.071990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.072981 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.072985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:17.073912 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:17.073924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.074555 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.074559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:17.074692 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:17.075302 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:17.075319 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:17.078979 (+ 3660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":270,"threads_started":1}
W20260808 02:58:17.079481 14028 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:17.070564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:48113 (local address 127.0.205.190:43153)
0808 02:58:17.070871 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:17.070879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:17.071071 (+ 192us) server_negotiation.cc:407] Connection header received
0808 02:58:17.071205 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:17.071209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:17.071273 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:17.071357 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:17.072115 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.072845 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.074039 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.074429 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.074521 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:17.079201 (+ 4680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:48113: BlockingRecv error: recv got EOF from 127.0.205.131:48113 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
Aug 08 02:58:17 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:17 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:17.212492 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:17.201247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:46037)
0808 02:58:17.201862 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:17.201953 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:17.202116 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:17.202503 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:17.202507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:17.202962 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:17.203209 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:17.203220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.204629 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.204634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:17.205969 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:17.205991 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.206791 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.206797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:17.207007 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:17.207735 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:17.207754 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:17.212076 (+ 4322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":325,"threads_started":1}
W20260808 02:58:17.212646 14028 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:17.201488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:46037 (local address 127.0.205.190:43153)
0808 02:58:17.201724 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:17.201734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:17.202040 (+ 306us) server_negotiation.cc:407] Connection header received
0808 02:58:17.202246 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:17.202252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:17.202340 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:17.202463 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:17.203416 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.204494 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.206144 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.206627 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.206774 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:17.212354 (+ 5580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:46037: BlockingRecv error: recv got EOF from 127.0.205.130:46037 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 02:58:17 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:17 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:17.931281 14031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:17.921538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:55165)
0808 02:58:17.922210 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:17.922228 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:17.922372 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:17.922793 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:17.922797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:17.923456 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:17.923667 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:17.923680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.924873 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.924879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:17.925820 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:17.925834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.926608 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.926613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:17.926818 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:17.927500 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:17.927519 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:17.931052 (+ 3533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":391,"threads_started":1}
W20260808 02:58:17.931550 14032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:17.921599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:55165 (local address 127.0.205.190:43153)
0808 02:58:17.922299 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:17.922306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:17.922327 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:58:17.922514 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:17.922520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:17.922626 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:17.922758 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:17.923812 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.924718 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.926011 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:17.926452 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:17.926597 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:17.931273 (+ 4676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:55165: BlockingRecv error: recv got EOF from 127.0.205.129:55165 (error 108)
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":438,"threads_started":1}
Aug 08 02:58:18 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:18 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:18.090227 14033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:18.080621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:54307)
0808 02:58:18.081161 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:18.081179 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:18.081301 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:18.081629 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:18.081632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:18.082084 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:18.082253 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:18.082262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.083268 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.083273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:18.084458 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:18.084476 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.085416 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.085423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:18.085567 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:18.086243 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:18.086261 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:18.089960 (+ 3699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":317,"threads_started":1}
W20260808 02:58:18.090370 14032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:18.080683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:54307 (local address 127.0.205.190:43153)
0808 02:58:18.080970 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:18.080978 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:18.081274 (+ 296us) server_negotiation.cc:407] Connection header received
0808 02:58:18.081417 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:18.081421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:18.081488 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:18.081575 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:18.082390 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.083126 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.084631 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.085249 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.085377 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:18.090151 (+ 4774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:54307: BlockingRecv error: recv got EOF from 127.0.205.131:54307 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 02:58:18 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:18 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:18.223549 14034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:18.213632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:58449)
0808 02:58:18.214245 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:18.214262 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:18.214383 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:18.214696 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:18.214700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:18.215170 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:18.215358 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:18.215368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.216777 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.216781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:18.217742 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:18.217754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.218582 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.218586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:18.218736 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:18.219445 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:18.219464 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:18.223321 (+ 3857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":358,"threads_started":1}
W20260808 02:58:18.223883 14032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:18.213910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:58449 (local address 127.0.205.190:43153)
0808 02:58:18.214106 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:18.214111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:18.214350 (+ 239us) server_negotiation.cc:407] Connection header received
0808 02:58:18.214475 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:18.214478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:18.214559 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:18.214650 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:18.215573 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.216577 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.217954 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.218425 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.218535 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:18.223568 (+ 5033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:58449: BlockingRecv error: recv got EOF from 127.0.205.130:58449 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 02:58:18 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:18 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:18.942389 14036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:18.932373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:43751)
0808 02:58:18.932980 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:18.933002 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:18.933222 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:18.933712 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:18.933719 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:18.934473 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:18.934740 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:18.934758 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.935865 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.935870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:18.936838 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:18.936851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.937562 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.937567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:18.937702 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:18.938385 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:18.938404 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:18.942048 (+ 3644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":331,"threads_started":1}
W20260808 02:58:18.942615 14037 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:18.932507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:43751 (local address 127.0.205.190:43153)
0808 02:58:18.933398 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:18.933403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:18.933420 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:58:18.933467 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:18.933471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:18.933528 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:18.933618 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:18.934927 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.935688 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.936985 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:18.937387 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:18.937483 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:18.942308 (+ 4825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:43751: BlockingRecv error: recv got EOF from 127.0.205.129:43751 (error 108)
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":649,"threads_started":1}
Aug 08 02:58:19 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:19 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:19.101346 14038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:19.091552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:59151)
0808 02:58:19.092142 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:19.092161 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:19.092313 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:19.092744 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:19.092748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:19.093254 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:19.093499 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:19.093510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.094794 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.094797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:19.095711 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:19.095723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.096526 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.096530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:19.096662 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:19.097361 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:19.097379 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:19.100997 (+ 3618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":329,"threads_started":1}
W20260808 02:58:19.101480 14037 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:19.091649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:59151 (local address 127.0.205.190:43153)
0808 02:58:19.091883 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:19.091890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:19.092225 (+ 335us) server_negotiation.cc:407] Connection header received
0808 02:58:19.092485 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:19.092491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:19.092574 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:19.092687 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:19.093703 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.094640 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.095865 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.096378 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.096501 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:19.101221 (+ 4720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:59151: BlockingRecv error: recv got EOF from 127.0.205.131:59151 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 02:58:19 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:19 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:19.234812 14039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:19.224651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:34389)
0808 02:58:19.225184 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:19.225210 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:19.225336 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:19.225673 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:19.225676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:19.226194 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:19.226470 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:19.226484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.227708 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.227712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:19.229104 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:19.229117 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.229838 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.229842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:19.230058 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:19.230712 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:19.230730 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:19.234469 (+ 3739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":297,"threads_started":1}
W20260808 02:58:19.234975 14037 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:19.224911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:34389 (local address 127.0.205.190:43153)
0808 02:58:19.225114 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:19.225118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:19.225295 (+ 177us) server_negotiation.cc:407] Connection header received
0808 02:58:19.225446 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:19.225450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:19.225528 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:19.225620 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:19.226618 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.227532 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.229242 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.229700 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.229796 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:19.234675 (+ 4879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:34389: BlockingRecv error: recv got EOF from 127.0.205.130:34389 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 02:58:19 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:19 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:19.954113 14040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:19.943583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:47915)
0808 02:58:19.944393 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:19.944419 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:19.944562 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:19.945312 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:19.945319 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:19.946000 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:19.946218 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:19.946228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.947384 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.947389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:19.948384 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:19.948397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.949177 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.949182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:19.949303 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:19.950115 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:19.950135 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:19.953770 (+ 3635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":346,"threads_started":1}
W20260808 02:58:19.954375 14043 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:19.943701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:47915 (local address 127.0.205.190:43153)
0808 02:58:19.944702 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:19.944710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:19.944732 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:58:19.944791 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:19.944794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:19.944855 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:19.944953 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:19.946361 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.947126 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.948568 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:19.949012 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:19.949123 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:19.954047 (+ 4924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:47915: BlockingRecv error: recv got EOF from 127.0.205.129:47915 (error 108)
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":727,"threads_started":1}
Aug 08 02:58:20 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:20 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:20.113085 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:20.102575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:34415)
0808 02:58:20.103182 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:20.103199 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:20.103355 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:20.103740 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:20.103748 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:20.104281 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:20.104460 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:20.104499 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.106026 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.106030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:20.107257 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:20.107271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.108144 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.108151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:20.108350 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:20.108998 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:20.109018 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:20.112826 (+ 3808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":350,"threads_started":1}
W20260808 02:58:20.113360 14043 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:20.102665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:34415 (local address 127.0.205.190:43153)
0808 02:58:20.102958 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:20.102963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:20.103286 (+ 323us) server_negotiation.cc:407] Connection header received
0808 02:58:20.103517 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:20.103521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:20.103588 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:20.103663 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:20.104675 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.105793 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.107446 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.107960 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.108119 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:20.113089 (+ 4970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:34415: BlockingRecv error: recv got EOF from 127.0.205.131:34415 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 08 02:58:20 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:20 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:20.246160 14046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:20.235955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:40445)
0808 02:58:20.236683 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:20.236700 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:20.236845 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:20.237184 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:20.237188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:20.237668 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:20.237854 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:20.237927 (+ 73us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.238975 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.238980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:20.240111 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:20.240124 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.241038 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.241045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:20.241224 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:20.241927 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:20.241954 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:20.245815 (+ 3861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":382,"threads_started":1}
W20260808 02:58:20.246393 14043 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:20.236122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:40445 (local address 127.0.205.190:43153)
0808 02:58:20.236363 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:20.236368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:20.236784 (+ 416us) server_negotiation.cc:407] Connection header received
0808 02:58:20.236953 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:20.236957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:20.237036 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:20.237130 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:20.238070 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.238794 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.240303 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.240848 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.240991 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:20.246114 (+ 5123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:40445: BlockingRecv error: recv got EOF from 127.0.205.130:40445 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 08 02:58:20 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:20 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:20.970666 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:20.955154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:44769)
0808 02:58:20.955914 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:20.955934 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:20.956073 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:20.956449 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:20.956452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:20.956879 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:20.957080 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:20.957091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.958220 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.958224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:20.959190 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:20.959202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.960085 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.960090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:20.960245 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:20.966238 (+ 5993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:20.966261 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:20.970397 (+ 4136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":461,"threads_started":1}
W20260808 02:58:20.970963 14049 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:20.955275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:44769 (local address 127.0.205.190:43153)
0808 02:58:20.955821 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:20.955826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:20.956024 (+ 198us) server_negotiation.cc:407] Connection header received
0808 02:58:20.956178 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:20.956181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:20.956245 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:20.956395 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:20.957224 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.958081 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.959366 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:20.959809 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:20.959988 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:20.970670 (+ 10682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:44769: BlockingRecv error: recv got EOF from 127.0.205.129:44769 (error 108)
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":312,"threads_started":1}
Aug 08 02:58:21 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:21 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:21.124257 14050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:21.114166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:55281)
0808 02:58:21.114854 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:21.114872 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:21.115039 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:21.115424 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:21.115429 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:21.115912 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:21.116090 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:21.116099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.117247 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.117252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:21.118249 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:21.118261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.119033 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.119039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:21.119207 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:21.119890 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:21.119909 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:21.123902 (+ 3993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":316,"threads_started":1}
W20260808 02:58:21.124452 14049 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:21.114370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:55281 (local address 127.0.205.190:43153)
0808 02:58:21.114609 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:21.114618 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:21.114966 (+ 348us) server_negotiation.cc:407] Connection header received
0808 02:58:21.115169 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:21.115178 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:21.115245 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:21.115356 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:21.116249 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.117092 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.118412 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.118867 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.119011 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:21.124180 (+ 5169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:55281: BlockingRecv error: recv got EOF from 127.0.205.131:55281 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 02:58:21 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:21 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:21.257211 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:21.247436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:38833)
0808 02:58:21.248062 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:21.248089 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:21.248240 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:21.248716 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:21.248721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:21.249289 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:21.249477 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:21.249487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.250787 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.250791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:21.251792 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:21.251804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.252562 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.252566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:21.252691 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:21.253382 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:21.253400 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:21.256955 (+ 3555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":317,"threads_started":1}
W20260808 02:58:21.257616 14049 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:21.247574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:38833 (local address 127.0.205.190:43153)
0808 02:58:21.247791 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:21.247798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:21.248219 (+ 421us) server_negotiation.cc:407] Connection header received
0808 02:58:21.248433 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:21.248439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:21.248549 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:21.248652 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:21.249626 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.250629 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.251943 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.252373 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.252490 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:21.257177 (+ 4687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:38833: BlockingRecv error: recv got EOF from 127.0.205.130:38833 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 02:58:21 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:21 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:21.981309 14052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:21.972001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:60961)
0808 02:58:21.972533 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:21.972550 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:21.972649 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:21.973248 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:21.973251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:21.973671 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:21.973824 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:21.973833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.974930 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.974934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:21.975907 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:21.975918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.976595 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.976599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:21.976717 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:21.977325 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:21.977341 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:21.981091 (+ 3750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":265,"threads_started":1}
W20260808 02:58:21.981639 14053 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:21.972075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:60961 (local address 127.0.205.190:43153)
0808 02:58:21.972960 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:21.972964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:21.972982 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:58:21.973033 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:21.973037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:21.973097 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:21.973218 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:21.974002 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.974803 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.976057 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:21.976466 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:21.976561 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:21.981316 (+ 4755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:60961: BlockingRecv error: recv got EOF from 127.0.205.129:60961 (error 108)
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":632,"threads_started":1}
Aug 08 02:58:22 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:22 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:22.135447 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:22.125454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:60517)
0808 02:58:22.126165 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:22.126189 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:22.126354 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:22.126799 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:22.126804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:22.127379 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:22.127637 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:22.127654 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.128693 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.128696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:22.129625 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:22.129636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.130431 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.130437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:22.130660 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:22.131518 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:22.131535 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:22.135162 (+ 3627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":422,"threads_started":1}
W20260808 02:58:22.135692 14053 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:22.125669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:60517 (local address 127.0.205.190:43153)
0808 02:58:22.125956 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:22.125962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:22.126291 (+ 329us) server_negotiation.cc:407] Connection header received
0808 02:58:22.126534 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:22.126540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:22.126627 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:22.126711 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:22.127803 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.128573 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.129745 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.130240 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.130367 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:22.135412 (+ 5045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:60517: BlockingRecv error: recv got EOF from 127.0.205.131:60517 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 08 02:58:22 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:22 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:22.268440 14056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:22.258534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:33237)
0808 02:58:22.259132 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:22.259150 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:22.259274 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:22.259624 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:22.259626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:22.260108 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:22.260264 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:22.260273 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.261492 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.261496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:22.262612 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:22.262625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.263537 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.263543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:22.263743 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:22.264348 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:22.264366 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:22.268091 (+ 3725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":335,"threads_started":1}
W20260808 02:58:22.268608 14053 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:22.258569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:33237 (local address 127.0.205.190:43153)
0808 02:58:22.258789 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:22.258794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:22.259240 (+ 446us) server_negotiation.cc:407] Connection header received
0808 02:58:22.259387 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:22.259399 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:22.259482 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:22.259581 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:22.260447 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.261334 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.262733 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.263354 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.263517 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:22.268318 (+ 4801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:33237: BlockingRecv error: recv got EOF from 127.0.205.130:33237 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 02:58:22 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:22 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:22.991299 14058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:22.982221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:56003)
0808 02:58:22.982801 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:22.982819 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:22.982932 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:22.983259 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:22.983262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:22.983656 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:22.983816 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:22.983825 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.984830 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.984834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:22.985818 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:22.985830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.986592 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.986596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:22.986716 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:22.987342 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:22.987360 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:22.990950 (+ 3590us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":340,"threads_started":1}
W20260808 02:58:22.991432 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:22.982312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:56003 (local address 127.0.205.190:43153)
0808 02:58:22.982872 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:22.982876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:22.982894 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:58:22.983044 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:22.983047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:22.983110 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:22.983209 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:22.983946 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.984707 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.986010 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:22.986445 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:22.986539 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:22.991161 (+ 4622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:56003: BlockingRecv error: recv got EOF from 127.0.205.129:56003 (error 108)
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":321,"threads_started":1}
Aug 08 02:58:23 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:23 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:23.146648 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:23.136649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:51003)
0808 02:58:23.137251 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:23.137267 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:23.137463 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:23.137861 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:23.137891 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:23.138422 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:23.138637 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:23.138651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.139755 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.139763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:23.140797 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:23.140810 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.141567 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.141573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:23.141754 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:23.142502 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:23.142520 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:23.146310 (+ 3790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":342,"threads_started":1}
W20260808 02:58:23.146777 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:23.136778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:51003 (local address 127.0.205.190:43153)
0808 02:58:23.137036 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:23.137046 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:23.137397 (+ 351us) server_negotiation.cc:407] Connection header received
0808 02:58:23.137601 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:23.137607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:23.137699 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:23.137817 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:23.138798 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.139590 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.140957 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.141396 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.141536 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:23.146526 (+ 4990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:51003: BlockingRecv error: recv got EOF from 127.0.205.131:51003 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 08 02:58:23 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:23 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:23.279147 14061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:23.269588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:53691)
0808 02:58:23.270174 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:23.270191 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:23.270310 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:23.270677 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:23.270680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:23.271080 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:23.271243 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:23.271252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.272279 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.272282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:23.273274 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:23.273286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.274047 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.274052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:23.274205 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:23.274971 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:23.274996 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:23.278887 (+ 3891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":368,"threads_started":1}
W20260808 02:58:23.279348 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:23.269680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:53691 (local address 127.0.205.190:43153)
0808 02:58:23.269961 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:23.269971 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:23.270285 (+ 314us) server_negotiation.cc:407] Connection header received
0808 02:58:23.270454 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:23.270459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:23.270542 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:23.270632 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:23.271376 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.272146 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.273398 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.273895 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.274027 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:23.279079 (+ 5052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:53691: BlockingRecv error: recv got EOF from 127.0.205.130:53691 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 02:58:23 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:23 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:24.002979 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:23.992495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:53797)
0808 02:58:23.993082 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:23.993099 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:23.993214 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:23.993645 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:23.993648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:23.994115 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:23.994434 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:23.994450 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.995805 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.995812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:23.997030 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:23.997044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.997761 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.997765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:23.997920 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:23.998553 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:23.998571 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:24.002631 (+ 4060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":341,"threads_started":1}
W20260808 02:58:24.003227 14063 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:23.992741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:53797 (local address 127.0.205.190:43153)
0808 02:58:23.993338 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:23.993343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:23.993361 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:58:23.993424 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:23.993428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:23.993488 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:23.993594 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:23.994604 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.995662 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.997170 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:23.997624 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:23.997723 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:24.002891 (+ 5168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:53797: BlockingRecv error: recv got EOF from 127.0.205.129:53797 (error 108)
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":344,"threads_started":1}
Aug 08 02:58:24 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:24 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:24.157713 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:24.147843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:51857)
0808 02:58:24.148413 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:24.148430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:24.148595 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:24.148975 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:24.148980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:24.149452 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:24.149618 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:24.149628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:24.150801 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:24.150807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:24.151819 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:24.151833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:24.152581 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:24.152587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:24.152757 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:24.153534 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:24.153558 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:24.157472 (+ 3914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":317,"threads_started":1}
W20260808 02:58:24.158097 14063 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:24.147874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:51857 (local address 127.0.205.190:43153)
0808 02:58:24.148121 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:24.148129 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:24.148522 (+ 393us) server_negotiation.cc:407] Connection header received
0808 02:58:24.148723 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:24.148732 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:24.148800 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:24.148940 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:24.149780 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:24.150650 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:24.151974 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:24.152411 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:24.152558 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:24.157661 (+ 5103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:51857: BlockingRecv error: recv got EOF from 127.0.205.131:51857 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 02:58:24 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:24 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:24.290114 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:24.280122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:58733)
0808 02:58:24.280689 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:24.280705 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:24.280826 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:24.281152 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:24.281155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:24.281610 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:24.281803 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:24.281812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:24.283129 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:24.283132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:24.284096 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:24.284108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:24.284824 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:24.284829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:24.284965 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:24.285567 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:24.285584 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:24.289760 (+ 4176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":343,"threads_started":1}
W20260808 02:58:24.290377 14063 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:24.280250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:58733 (local address 127.0.205.190:43153)
0808 02:58:24.280425 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:24.280430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:24.280788 (+ 358us) server_negotiation.cc:407] Connection header received
0808 02:58:24.280936 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:24.280940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:24.281017 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:24.281102 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:24.281988 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:24.282972 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:24.284246 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:24.284683 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:24.284783 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:24.290043 (+ 5260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:58733: BlockingRecv error: recv got EOF from 127.0.205.130:58733 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 02:58:25 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:25 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:25.014617 14068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:25.004035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:40245)
0808 02:58:25.004670 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:25.004692 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:25.004850 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:25.005253 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:25.005257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:25.005721 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:25.005989 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:25.006002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.007564 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.007570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:25.008916 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:25.008930 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.009724 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.009728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:25.009861 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:25.010674 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:25.010694 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:25.014356 (+ 3662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":345,"threads_started":1}
W20260808 02:58:25.014763 14069 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:25.004123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:40245 (local address 127.0.205.190:43153)
0808 02:58:25.004693 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:25.004699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:25.004781 (+ 82us) server_negotiation.cc:407] Connection header received
0808 02:58:25.005008 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:25.005014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:25.005098 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:25.005212 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:25.006195 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.007397 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.009055 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.009561 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.009699 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:25.014545 (+ 4846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:40245: BlockingRecv error: recv got EOF from 127.0.205.129:40245 (error 108)
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":345,"threads_started":1}
Aug 08 02:58:25 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:25 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:25.168804 14070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:25.158724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:38557)
0808 02:58:25.159243 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:25.159260 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:25.159412 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:25.159715 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:25.159718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:25.160141 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:25.160360 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:25.160370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.161428 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.161433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:25.162597 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:25.162612 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.163357 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.163362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:25.163513 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:25.164300 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:25.164321 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:25.168471 (+ 4150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":316,"threads_started":1}
W20260808 02:58:25.168886 14069 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:25.158799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:38557 (local address 127.0.205.190:43153)
0808 02:58:25.159059 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:25.159067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:25.159350 (+ 283us) server_negotiation.cc:407] Connection header received
0808 02:58:25.159518 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:25.159522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:25.159586 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:25.159671 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:25.160491 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.161256 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.162760 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.163179 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.163327 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:25.168635 (+ 5308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:38557: BlockingRecv error: recv got EOF from 127.0.205.131:38557 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 08 02:58:25 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:25 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:25.300778 14071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:25.291181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:34423)
0808 02:58:25.291821 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:25.291838 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:25.291961 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:25.292278 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:25.292282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:25.292713 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:25.292890 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:25.292901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.294006 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.294010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:25.294971 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:25.294983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.295786 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.295790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:25.295923 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:25.296604 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:25.296622 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:25.300476 (+ 3854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":397,"threads_started":1}
W20260808 02:58:25.301055 14069 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:25.291267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:34423 (local address 127.0.205.190:43153)
0808 02:58:25.291632 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:25.291639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:25.291925 (+ 286us) server_negotiation.cc:407] Connection header received
0808 02:58:25.292063 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:25.292067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:25.292146 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:25.292232 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:25.293035 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.293849 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.295134 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:25.295622 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:25.295730 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:25.300681 (+ 4951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:34423: BlockingRecv error: recv got EOF from 127.0.205.130:34423 (error 108)
Metrics: {"server-negotiator.queue_time_us":187}
Aug 08 02:58:26 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:26 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:26.029455 14073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:26.015836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:49679)
0808 02:58:26.016437 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:26.016460 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:26.016597 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:26.017013 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:26.017018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:26.017551 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:26.017706 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:26.017716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.018817 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.018823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:26.019766 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:26.019779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.020538 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.020544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:26.020746 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:26.025250 (+ 4504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:26.025270 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:26.029184 (+ 3914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":306,"threads_started":1}
W20260808 02:58:26.030064 14074 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:26.015925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:49679 (local address 127.0.205.190:43153)
0808 02:58:26.016552 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:26.016557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:26.016576 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:58:26.016742 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:26.016748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:26.016834 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:26.016980 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:26.017915 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.018658 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.019930 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.020382 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.020523 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:26.029466 (+ 8943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:49679: BlockingRecv error: recv got EOF from 127.0.205.129:49679 (error 108)
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":380,"threads_started":1}
Aug 08 02:58:26 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:26 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:26.179862 14076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:26.170023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:41607)
0808 02:58:26.170643 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:26.170660 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:26.170793 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:26.171162 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:26.171166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:26.171600 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:26.171779 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:26.171788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.172832 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.172837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:26.173841 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:26.173862 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.174663 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.174669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:26.174804 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:26.175693 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:26.175720 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:26.179593 (+ 3873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":339,"threads_started":1}
W20260808 02:58:26.180027 14074 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:26.170014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:41607 (local address 127.0.205.190:43153)
0808 02:58:26.170275 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:26.170286 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:26.170771 (+ 485us) server_negotiation.cc:407] Connection header received
0808 02:58:26.170912 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:26.170916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:26.170983 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:26.171072 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:26.171918 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.172667 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.174049 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.174457 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.174554 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:26.179806 (+ 5252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:41607: BlockingRecv error: recv got EOF from 127.0.205.131:41607 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 08 02:58:26 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:26 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:26.312357 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:26.301950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:44865)
0808 02:58:26.302520 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:26.302540 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:26.302707 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:26.303123 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:26.303126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:26.303563 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:26.303760 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:26.303801 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.305129 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.305135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:26.306523 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:26.306537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.307461 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.307465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:26.307590 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:26.308268 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:26.308286 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:26.312041 (+ 3755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":353,"threads_started":1}
W20260808 02:58:26.312565 14074 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:26.302231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:44865 (local address 127.0.205.190:43153)
0808 02:58:26.302415 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:26.302419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:26.302640 (+ 221us) server_negotiation.cc:407] Connection header received
0808 02:58:26.302836 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:26.302844 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:26.302959 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:26.303094 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:26.303978 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.304928 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.306719 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:26.307319 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:26.307431 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:26.312268 (+ 4837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:44865: BlockingRecv error: recv got EOF from 127.0.205.130:44865 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 02:58:27 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:27 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.129@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:27.040416 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:27.030716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.129:43875)
0808 02:58:27.031279 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:27.031297 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:27.031407 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:27.031696 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:27.031699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:27.032095 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:27.032282 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:27.032291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.033396 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.033400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:27.034372 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:27.034383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.035059 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.035063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:27.035191 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:27.035876 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:27.035894 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:27.040137 (+ 4243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":301,"threads_started":1}
W20260808 02:58:27.040623 14079 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:27.030883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:43875 (local address 127.0.205.190:43153)
0808 02:58:27.031418 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:27.031422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:27.031440 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:58:27.031492 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:27.031495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:27.031553 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:27.031650 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:27.032446 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.033217 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.034499 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.034930 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.035026 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:27.040355 (+ 5329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.129:43875: BlockingRecv error: recv got EOF from 127.0.205.129:43875 (error 108)
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":304,"threads_started":1}
Aug 08 02:58:27 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:27 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.131@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:27.190596 14080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:27.180960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.131:56667)
0808 02:58:27.181450 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:27.181467 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:27.181591 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:27.181941 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:27.181945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:27.182483 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:27.182641 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:27.182650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.183942 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.183948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:27.184895 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:27.184907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.185741 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.185747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:27.185984 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:27.186569 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:27.186587 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:27.190243 (+ 3656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":258,"threads_started":1}
W20260808 02:58:27.190699 14079 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:27.181060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:56667 (local address 127.0.205.190:43153)
0808 02:58:27.181447 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:27.181454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:27.181547 (+ 93us) server_negotiation.cc:407] Connection header received
0808 02:58:27.181688 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:27.181692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:27.181756 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:27.181839 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:27.182833 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.183771 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.185057 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.185575 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.185714 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:27.190471 (+ 4757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.131:56667: BlockingRecv error: recv got EOF from 127.0.205.131:56667 (error 108)
Metrics: {"server-negotiator.queue_time_us":208}
Aug 08 02:58:27 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:58:27 dist-test-slave-6ntk krb5kdc[12551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.205.130@KRBTEST.COM for oryx/127.0.205.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:58:27.324072 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:27.313857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43153 (local address 127.0.205.130:41987)
0808 02:58:27.314501 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:27.314518 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:27.314634 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:27.314945 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:27.314948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:27.315446 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:27.315700 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:27.315715 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.316899 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.316905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:27.318085 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:27.318104 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.318994 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.319001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:27.319172 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:27.319935 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:27.319953 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:27.323797 (+ 3844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.205.190:43153: Server oryx/127.0.205.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":339,"threads_started":1}
W20260808 02:58:27.324368 14079 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:27.314022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:41987 (local address 127.0.205.190:43153)
0808 02:58:27.314211 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:27.314217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:27.314596 (+ 379us) server_negotiation.cc:407] Connection header received
0808 02:58:27.314745 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:27.314748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:27.314810 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:27.314899 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:27.315867 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.316729 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.318232 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.318826 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.318964 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:27.324051 (+ 5087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.205.130:41987: BlockingRecv error: recv got EOF from 127.0.205.130:41987 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:58:27.893651 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 13413
I20260808 02:58:27.906816 14085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:27.894983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41138 (local address 127.0.205.129:39581)
0808 02:58:27.895661 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:27.895669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:27.895697 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:58:27.895777 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:27.895784 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:27.896039 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:27.896225 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:27.897764 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.898885 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.900108 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.900662 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.900768 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:27.903552 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:27.903560 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:27.903575 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:27.903630 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:27.906263 (+ 2633us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.205.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 02:58:27.906600 (+ 337us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41138: Request ticket server kudu/127.0.205.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":384,"threads_started":1}
W20260808 02:58:27.907012 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 13413: Not authorized: Client connection negotiation failed: client connection to 127.0.205.129:39581: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.205.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 02:58:27.907076 14085 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41138: Request ticket server kudu/127.0.205.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 02:58:27.907263 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 13413
I20260808 02:58:27.938115 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 13619
I20260808 02:58:27.951236 14086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:27.939203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.0.205.130:38765)
0808 02:58:27.939829 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:27.939838 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:27.939865 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:58:27.939931 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:27.939938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:27.940188 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:27.940368 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:27.942381 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.943426 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.944660 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.945207 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.945310 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:27.947894 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:27.947903 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:27.947918 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:27.947956 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:27.950660 (+ 2704us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.205.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 02:58:27.950944 (+ 284us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44196: Request ticket server kudu/127.0.205.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":327,"threads_started":1}
W20260808 02:58:27.951352 14086 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44196: Request ticket server kudu/127.0.205.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 02:58:27.951350 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 13619: Not authorized: Client connection negotiation failed: client connection to 127.0.205.130:38765: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.205.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 02:58:27.951536 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 13619
I20260808 02:58:27.980172 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 13817
I20260808 02:58:27.993471 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:27.981421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55904 (local address 127.0.205.131:42301)
0808 02:58:27.982165 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:27.982175 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:27.982208 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:58:27.982293 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:27.982302 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:27.982604 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:27.982825 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:27.984197 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.985675 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.986964 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:27.987516 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:27.987620 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:27.990318 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:27.990327 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:27.990344 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:27.990379 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:27.992925 (+ 2546us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.205.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 02:58:27.993231 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55904: Request ticket server kudu/127.0.205.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":339,"threads_started":1}
W20260808 02:58:27.993613 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 13817: Not authorized: Client connection negotiation failed: client connection to 127.0.205.131:42301: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.0.205.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 02:58:27.993726 14090 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55904: Request ticket server kudu/127.0.205.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 02:58:27.993816 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 13817
I20260808 02:58:28.013494 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 13317
I20260808 02:58:28.014912 13404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:28.138804 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 13317
2026-08-08T02:58:28Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22387 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T02:58:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:58:28Z Disabled control of system clock
I20260808 02:58:28.244441 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:45837
--webserver_interface=127.0.205.190
--webserver_port=0
--builtin_ntp_servers=127.0.205.148:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:45837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 02:58:28.599766 14099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:28.600070 14099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:28.600250 14099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:28.607764 14099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:58:28.607884 14099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:28.607951 14099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:28.608062 14099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:58:28.608142 14099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:58:28.617934 14099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:45837
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:45837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:28.619372 14099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:28.621269 14099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:28.634012 14109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:28.634044 14112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:28.634081 14106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:28.634311 14099 server_base.cc:1034] running on GCE node
I20260808 02:58:28.635632 14099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:28.637246 14099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:28.638541 14099 hybrid_clock.cc:648] HybridClock initialized: now 1786157908638482 us; error 46 us; skew 500 ppm
I20260808 02:58:28.638976 14099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:58:28.641757 14099 webserver.cc:460] Webserver started at http://127.0.205.190:43101/ using document root <none> and password file <none>
I20260808 02:58:28.642571 14099 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:28.642717 14099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:28.643082 14099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:28.646298 14099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "a62b0ec0ff964ca99a43bf401752b2a0"
format_stamp: "Formatted at 2026-08-08 02:58:28 on dist-test-slave-6ntk"
server_key: "7e5a98ce592ed757b3928266ab7454d0"
server_key_iv: "e20918f8e9ed7d99b756f8d9d571a97b"
server_key_version: "encryptionkey@0"
I20260808 02:58:28.647156 14099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "a62b0ec0ff964ca99a43bf401752b2a0"
format_stamp: "Formatted at 2026-08-08 02:58:28 on dist-test-slave-6ntk"
server_key: "7e5a98ce592ed757b3928266ab7454d0"
server_key_iv: "e20918f8e9ed7d99b756f8d9d571a97b"
server_key_version: "encryptionkey@0"
I20260808 02:58:28.652834 14099 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 02:58:28.656569 14119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:28.658034 14099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:58:28.658231 14099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "a62b0ec0ff964ca99a43bf401752b2a0"
format_stamp: "Formatted at 2026-08-08 02:58:28 on dist-test-slave-6ntk"
server_key: "7e5a98ce592ed757b3928266ab7454d0"
server_key_iv: "e20918f8e9ed7d99b756f8d9d571a97b"
server_key_version: "encryptionkey@0"
I20260808 02:58:28.658489 14099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:28.686434 14099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:28.687358 14099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:28.687687 14099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:28.707589 14099 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:45837
I20260808 02:58:28.707608 14190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:45837 every 8 connection(s)
I20260808 02:58:28.709030 14099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:58:28.711594 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 14099
I20260808 02:58:28.711817 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:58:28.712528 822 external_mini_cluster.cc:1428] Setting key 5470b2e47304fd7d99b8a84c815e7efa
I20260808 02:58:28.713248 14192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:28.729161 14195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:28.714861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49478 (local address 127.0.205.190:45837)
0808 02:58:28.715991 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:28.716005 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:28.716058 (+ 53us) server_negotiation.cc:407] Connection header received
0808 02:58:28.718493 (+ 2435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:28.718526 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:28.719173 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:28.719650 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:28.720231 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:28.722283 (+ 2052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:28.723690 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:28.724786 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:28.724973 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:28.726193 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:28.726208 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:28.726228 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:28.727034 (+ 806us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:28.727149 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:28.727185 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:28.727295 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:28.727508 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 02:58:28.727842 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":373,"threads_started":1}
I20260808 02:58:28.729477 14192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0: Bootstrap starting.
I20260808 02:58:28.732879 14192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:28.733969 14192 log.cc:824] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0: Log is configured to *not* fsync() on all Append() calls
I20260808 02:58:28.737131 14192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0: No bootstrap required, opened a new log
I20260808 02:58:28.744923 14192 raft_consensus.cc:348] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62b0ec0ff964ca99a43bf401752b2a0" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45837 } }
I20260808 02:58:28.745390 14192 raft_consensus.cc:374] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:28.745505 14192 raft_consensus.cc:729] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a62b0ec0ff964ca99a43bf401752b2a0, State: Initialized, Role: FOLLOWER
I20260808 02:58:28.746263 14192 consensus_queue.cc:260] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62b0ec0ff964ca99a43bf401752b2a0" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45837 } }
I20260808 02:58:28.746577 14192 raft_consensus.cc:388] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:58:28.746745 14192 raft_consensus.cc:482] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:58:28.746950 14192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:28.749239 14192 raft_consensus.cc:504] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62b0ec0ff964ca99a43bf401752b2a0" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45837 } }
I20260808 02:58:28.749804 14192 leader_election.cc:302] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a62b0ec0ff964ca99a43bf401752b2a0; no voters:
I20260808 02:58:28.750510 14192 leader_election.cc:288] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:58:28.750697 14197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:58:28.751473 14197 raft_consensus.cc:686] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [term 1 LEADER]: Becoming Leader. State: Replica: a62b0ec0ff964ca99a43bf401752b2a0, State: Running, Role: LEADER
I20260808 02:58:28.752084 14197 consensus_queue.cc:237] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62b0ec0ff964ca99a43bf401752b2a0" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45837 } }
I20260808 02:58:28.752712 14192 sys_catalog.cc:564] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:58:28.755259 14199 sys_catalog.cc:455] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a62b0ec0ff964ca99a43bf401752b2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62b0ec0ff964ca99a43bf401752b2a0" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45837 } } }
I20260808 02:58:28.755705 14199 sys_catalog.cc:458] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:58:28.760164 14200 sys_catalog.cc:455] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a62b0ec0ff964ca99a43bf401752b2a0. Latest consensus state: current_term: 1 leader_uuid: "a62b0ec0ff964ca99a43bf401752b2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62b0ec0ff964ca99a43bf401752b2a0" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45837 } } }
I20260808 02:58:28.760614 14200 sys_catalog.cc:458] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0 [sys.catalog]: This master's current role is: LEADER
I20260808 02:58:28.762583 14207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:58:28.768153 14207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:58:28.776942 14207 catalog_manager.cc:1347] Generated new cluster ID: 1da7d0d6432a4d8ab8e8cc0831b4d497
I20260808 02:58:28.777091 14207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:58:28.801880 14207 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:58:28.803833 14207 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:58:28.818210 14207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a62b0ec0ff964ca99a43bf401752b2a0: Generated new TSK 0
I20260808 02:58:28.819420 14207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:58:28.827965 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45837
--builtin_ntp_servers=127.0.205.148:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 02:58:29.189057 14222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:29.189354 14222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:29.189517 14222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:29.197036 14222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:29.197176 14222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:29.197356 14222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:58:29.206950 14222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:29.208339 14222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:29.210335 14222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:29.224546 14227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:29.224611 14228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:29.224587 14230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:29.225153 14222 server_base.cc:1034] running on GCE node
I20260808 02:58:29.226015 14222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:29.226984 14222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:29.228196 14222 hybrid_clock.cc:648] HybridClock initialized: now 1786157909228126 us; error 69 us; skew 500 ppm
I20260808 02:58:29.228602 14222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:58:29.231063 14222 webserver.cc:460] Webserver started at http://127.0.205.129:46731/ using document root <none> and password file <none>
I20260808 02:58:29.231740 14222 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:29.231840 14222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:29.232126 14222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:29.235051 14222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "6b32ea5d9ac345d8bea532bb8e6dcd1d"
format_stamp: "Formatted at 2026-08-08 02:58:29 on dist-test-slave-6ntk"
server_key: "ad0e37b2d5365c1dbc6c89cd921acb2e"
server_key_iv: "65aacfebaf6fa1b9113e42edd4bfe007"
server_key_version: "encryptionkey@0"
I20260808 02:58:29.235910 14222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "6b32ea5d9ac345d8bea532bb8e6dcd1d"
format_stamp: "Formatted at 2026-08-08 02:58:29 on dist-test-slave-6ntk"
server_key: "ad0e37b2d5365c1dbc6c89cd921acb2e"
server_key_iv: "65aacfebaf6fa1b9113e42edd4bfe007"
server_key_version: "encryptionkey@0"
I20260808 02:58:29.241148 14222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 02:58:29.244844 14239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:29.246318 14222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:58:29.246518 14222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "6b32ea5d9ac345d8bea532bb8e6dcd1d"
format_stamp: "Formatted at 2026-08-08 02:58:29 on dist-test-slave-6ntk"
server_key: "ad0e37b2d5365c1dbc6c89cd921acb2e"
server_key_iv: "65aacfebaf6fa1b9113e42edd4bfe007"
server_key_version: "encryptionkey@0"
I20260808 02:58:29.246755 14222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:29.268306 14222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:29.269272 14222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:29.269621 14222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:29.270689 14222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:29.272248 14222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:29.272392 14222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:29.272522 14222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:29.272612 14222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:29.307090 14222 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:35687
I20260808 02:58:29.307185 14408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:35687 every 8 connection(s)
I20260808 02:58:29.308454 14222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:58:29.309016 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 14222
I20260808 02:58:29.309247 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:58:29.310171 822 external_mini_cluster.cc:1428] Setting key 87241d98ff1c76379646a3e7b830e104
I20260808 02:58:29.313207 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45837
--builtin_ntp_servers=127.0.205.148:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 02:58:29.322754 14411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:29.311360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:59397 (local address 127.0.205.190:45837)
0808 02:58:29.311878 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:29.311884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:29.313290 (+ 1406us) server_negotiation.cc:407] Connection header received
0808 02:58:29.315198 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:29.315206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:29.315297 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:29.315424 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:29.316486 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:29.317671 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:29.319401 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:29.320130 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:29.320282 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:29.321700 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:29.321709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:29.321716 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:29.321743 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:29.321795 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:29.321815 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:29.321958 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:29.322184 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 02:58:29.322299 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":262,"threads_started":1}
I20260808 02:58:29.323607 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:29.312013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:45837 (local address 127.0.205.129:59397)
0808 02:58:29.313103 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:29.313156 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:29.314915 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:29.315634 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:29.315642 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:29.315729 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:29.316308 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:29.316325 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:29.317928 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:29.317933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:29.319244 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:29.319257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:29.320353 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:29.320357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:29.320562 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:29.321341 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:58:29.321344 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:29.321388 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:58:29.321480 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:29.321965 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:29.321974 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:29.321980 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:58:29.322139 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 02:58:29.322449 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":364,"threads_started":1}
I20260808 02:58:29.325374 14409 heartbeater.cc:347] Connected to a master server at 127.0.205.190:45837
I20260808 02:58:29.325697 14409 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:29.326627 14409 heartbeater.cc:511] Master 127.0.205.190:45837 requested a full tablet report, sending...
I20260808 02:58:29.328805 14142 ts_manager.cc:194] Registered new tserver with Master: 6b32ea5d9ac345d8bea532bb8e6dcd1d (127.0.205.129:35687)
I20260808 02:58:29.330404 14142 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.129:59397
W20260808 02:58:29.684782 14413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:29.685150 14413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:29.685329 14413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:29.693002 14413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:29.693195 14413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:29.693392 14413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:58:29.703065 14413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:29.704478 14413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:29.706434 14413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:29.720882 14421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:29.720906 14424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:29.721468 14420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:29.722088 14413 server_base.cc:1034] running on GCE node
I20260808 02:58:29.722728 14413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:29.723856 14413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:29.725100 14413 hybrid_clock.cc:648] HybridClock initialized: now 1786157909725034 us; error 57 us; skew 500 ppm
I20260808 02:58:29.725414 14413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:58:29.727885 14413 webserver.cc:460] Webserver started at http://127.0.205.130:33583/ using document root <none> and password file <none>
I20260808 02:58:29.728632 14413 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:29.728730 14413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:29.729030 14413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:29.732062 14413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "f0e1cf43c2d04153852ceadd92966b10"
format_stamp: "Formatted at 2026-08-08 02:58:29 on dist-test-slave-6ntk"
server_key: "aa8ec360fe367847f699adcb013c78d8"
server_key_iv: "ae516d8776e390a1db68976da5f20bc3"
server_key_version: "encryptionkey@0"
I20260808 02:58:29.732836 14413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "f0e1cf43c2d04153852ceadd92966b10"
format_stamp: "Formatted at 2026-08-08 02:58:29 on dist-test-slave-6ntk"
server_key: "aa8ec360fe367847f699adcb013c78d8"
server_key_iv: "ae516d8776e390a1db68976da5f20bc3"
server_key_version: "encryptionkey@0"
I20260808 02:58:29.738225 14413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 02:58:29.741837 14430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:29.743362 14413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:58:29.743571 14413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "f0e1cf43c2d04153852ceadd92966b10"
format_stamp: "Formatted at 2026-08-08 02:58:29 on dist-test-slave-6ntk"
server_key: "aa8ec360fe367847f699adcb013c78d8"
server_key_iv: "ae516d8776e390a1db68976da5f20bc3"
server_key_version: "encryptionkey@0"
I20260808 02:58:29.743789 14413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:29.775240 14413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:29.776211 14413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:29.776535 14413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:29.777596 14413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:29.779235 14413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:29.779330 14413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:29.779469 14413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:29.779554 14413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:29.814735 14413 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:37017
I20260808 02:58:29.814782 14590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:37017 every 8 connection(s)
I20260808 02:58:29.816128 14413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:58:29.825202 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 14413
I20260808 02:58:29.825350 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:58:29.825937 822 external_mini_cluster.cc:1428] Setting key 80a4e94ad41c526ddcb387e12b1652f2
I20260808 02:58:29.829113 14411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:29.818774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:38353 (local address 127.0.205.190:45837)
0808 02:58:29.818973 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:29.818979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:29.820425 (+ 1446us) server_negotiation.cc:407] Connection header received
0808 02:58:29.822251 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:29.822255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:29.822326 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:29.822420 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:29.823454 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:29.824239 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:29.825977 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:29.826591 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:29.826698 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:29.828212 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:29.828219 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:29.828224 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:29.828246 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:29.828287 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:29.828303 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:29.828403 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:29.828620 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 02:58:29.828719 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:58:29.829136 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45837
--builtin_ntp_servers=127.0.205.148:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 02:58:29.830252 14593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:29.819387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:45837 (local address 127.0.205.130:38353)
0808 02:58:29.820283 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:29.820325 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:29.821995 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:29.822623 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:29.822631 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:29.822715 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:29.823299 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:29.823314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:29.824389 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:29.824392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:29.825763 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:29.825776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:29.826857 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:29.826866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:29.827069 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:29.827837 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:58:29.827841 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:29.827900 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:58:29.828003 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:29.828410 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:29.828418 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:29.828424 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:58:29.828573 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 02:58:29.828889 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":267,"threads_started":1}
I20260808 02:58:29.831859 14591 heartbeater.cc:347] Connected to a master server at 127.0.205.190:45837
I20260808 02:58:29.832161 14591 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:29.832930 14591 heartbeater.cc:511] Master 127.0.205.190:45837 requested a full tablet report, sending...
I20260808 02:58:29.834512 14142 ts_manager.cc:194] Registered new tserver with Master: f0e1cf43c2d04153852ceadd92966b10 (127.0.205.130:37017)
I20260808 02:58:29.835709 14142 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.130:38353
W20260808 02:58:30.177548 14594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:30.177934 14594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:30.178105 14594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:30.185524 14594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:30.185662 14594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:30.185931 14594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:58:30.198621 14594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:30.199970 14594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:30.201805 14594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:30.215361 14600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:30.215448 14601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:30.215476 14604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:30.216656 14594 server_base.cc:1034] running on GCE node
I20260808 02:58:30.217335 14594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:30.218423 14594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:30.219676 14594 hybrid_clock.cc:648] HybridClock initialized: now 1786157910219619 us; error 50 us; skew 500 ppm
I20260808 02:58:30.220024 14594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:58:30.222470 14594 webserver.cc:460] Webserver started at http://127.0.205.131:44285/ using document root <none> and password file <none>
I20260808 02:58:30.223196 14594 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:30.223346 14594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:30.223699 14594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:30.230020 14594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "26ab4bcc463041649ce8da5e9c883098"
format_stamp: "Formatted at 2026-08-08 02:58:30 on dist-test-slave-6ntk"
server_key: "a5de426358f97633975b7efbdb1f060c"
server_key_iv: "f5aace0c52bed7309908a406b5e74147"
server_key_version: "encryptionkey@0"
I20260808 02:58:30.230883 14594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "26ab4bcc463041649ce8da5e9c883098"
format_stamp: "Formatted at 2026-08-08 02:58:30 on dist-test-slave-6ntk"
server_key: "a5de426358f97633975b7efbdb1f060c"
server_key_iv: "f5aace0c52bed7309908a406b5e74147"
server_key_version: "encryptionkey@0"
I20260808 02:58:30.236402 14594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260808 02:58:30.240231 14611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:30.241808 14594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:58:30.242036 14594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "26ab4bcc463041649ce8da5e9c883098"
format_stamp: "Formatted at 2026-08-08 02:58:30 on dist-test-slave-6ntk"
server_key: "a5de426358f97633975b7efbdb1f060c"
server_key_iv: "f5aace0c52bed7309908a406b5e74147"
server_key_version: "encryptionkey@0"
I20260808 02:58:30.242304 14594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:30.265393 14594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:30.266418 14594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:30.266755 14594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:30.267814 14594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:30.269567 14594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:30.269692 14594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:30.269834 14594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:30.269958 14594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 02:58:30.306737 14594 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:35263
I20260808 02:58:30.306710 14784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:35263 every 8 connection(s)
I20260808 02:58:30.308490 14594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:58:30.309450 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 14594
I20260808 02:58:30.309628 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:58:30.310300 822 external_mini_cluster.cc:1428] Setting key 8ff4684972d35c19bd7154d1f1352c26
I20260808 02:58:30.324252 14411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:30.312798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:56707 (local address 127.0.205.190:45837)
0808 02:58:30.312985 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:30.312991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:30.315281 (+ 2290us) server_negotiation.cc:407] Connection header received
0808 02:58:30.317472 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:30.317478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:30.317569 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:30.317701 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:30.318871 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:30.319627 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:30.321219 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:30.321675 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:30.321824 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:30.323369 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:30.323374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:30.323377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:30.323396 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:30.323429 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:30.323442 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:30.323545 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:30.323782 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 02:58:30.323889 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 02:58:30.325446 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:30.312163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:45837 (local address 127.0.205.131:56707)
0808 02:58:30.315110 (+ 2947us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:30.315172 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:30.317186 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:30.318009 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:30.318017 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:30.318107 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:30.318684 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:30.318700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:30.319804 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:30.319813 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:30.321065 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:30.321078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:30.321973 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:30.322011 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:30.322243 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:30.323005 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:58:30.323008 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:30.323050 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:58:30.323138 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:30.323545 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:30.323558 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:30.323568 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 02:58:30.323756 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 02:58:30.324178 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2379,"thread_start_us":2129,"threads_started":1}
I20260808 02:58:30.327065 14785 heartbeater.cc:347] Connected to a master server at 127.0.205.190:45837
I20260808 02:58:30.327349 14785 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:30.328116 14785 heartbeater.cc:511] Master 127.0.205.190:45837 requested a full tablet report, sending...
I20260808 02:58:30.329593 14142 ts_manager.cc:194] Registered new tserver with Master: 26ab4bcc463041649ce8da5e9c883098 (127.0.205.131:35263)
I20260808 02:58:30.330456 14142 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.131:56707
I20260808 02:58:30.333451 14409 heartbeater.cc:503] Master 127.0.205.190:45837 was elected leader, sending a full tablet report...
I20260808 02:58:30.339460 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:58:30.346294 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:30.344432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49492 (local address 127.0.205.190:45837)
0808 02:58:30.344601 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:30.344606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:30.345133 (+ 527us) server_negotiation.cc:407] Connection header received
0808 02:58:30.345325 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:30.345329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:30.345390 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:30.345478 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:30.346027 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49492: BlockingRecv error: recv got EOF from 127.0.0.1:49492 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 02:58:30.347244 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 14222
I20260808 02:58:30.356045 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:30.348323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39416 (local address 127.0.205.129:35687)
0808 02:58:30.348965 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:30.348972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:30.349000 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:58:30.349065 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:30.349071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:30.349287 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:30.349433 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:30.349794 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:30.351057 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:30.352550 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:30.353204 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:30.353310 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:30.354596 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:30.354605 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:30.354619 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:30.355218 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:30.355273 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:30.355294 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:30.355391 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:30.355598 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 02:58:30.355724 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":324,"threads_started":1}
I20260808 02:58:30.357348 14404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:30.443220 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 14222
I20260808 02:58:30.466785 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 14413
I20260808 02:58:30.475253 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:30.467954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54418 (local address 127.0.205.130:37017)
0808 02:58:30.468543 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:30.468550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:30.468578 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:58:30.468658 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:30.468664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:30.469022 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:30.469163 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:30.469583 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:30.470760 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:30.472091 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:30.472692 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:30.472802 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:30.473797 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:30.473805 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:30.473819 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:30.474520 (+ 701us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:30.474579 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:30.474599 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:30.474693 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:30.474831 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 02:58:30.474953 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":324,"threads_started":1}
I20260808 02:58:30.476629 14586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:30.560914 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 14413
I20260808 02:58:30.583504 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 14594
I20260808 02:58:30.592123 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:30.584610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57304 (local address 127.0.205.131:35263)
0808 02:58:30.585310 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:30.585317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:30.585349 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:58:30.585412 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:30.585418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:30.585641 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:30.585777 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:30.586282 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:30.587372 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:30.588609 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:30.589258 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:30.589381 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:30.590622 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:30.590631 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:30.590645 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:30.591281 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:30.591332 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:30.591354 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:30.591469 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:30.591631 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 02:58:30.591763 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":376,"threads_started":1}
I20260808 02:58:30.593631 14779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:30.677824 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 14594
I20260808 02:58:30.703413 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 14099
I20260808 02:58:30.704687 14183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:30.810143 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 14099
2026-08-08T02:58:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2658 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T02:58:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:58:30Z Disabled control of system clock
I20260808 02:58:30.901281 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:39407
--webserver_interface=127.0.205.190
--webserver_port=0
--builtin_ntp_servers=127.0.205.148:36489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:39407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:58:31.250039 14812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:31.250339 14812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:31.250519 14812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:31.257753 14812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:58:31.257860 14812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:31.257962 14812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:31.258047 14812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:58:31.258108 14812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:58:31.267549 14812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:36489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:39407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:39407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:31.269013 14812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:31.271028 14812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:31.283784 14831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:31.283862 14824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:31.283908 14822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:31.284689 14812 server_base.cc:1034] running on GCE node
I20260808 02:58:31.285606 14812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:31.287164 14812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:31.288427 14812 hybrid_clock.cc:648] HybridClock initialized: now 1786157911288379 us; error 61 us; skew 500 ppm
I20260808 02:58:31.288841 14812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:58:31.296013 14812 webserver.cc:460] Webserver started at http://127.0.205.190:36473/ using document root <none> and password file <none>
I20260808 02:58:31.296823 14812 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:31.296974 14812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:31.297326 14812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:31.300491 14812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "fded01e6a1a24e79ae5b14a4285ff58a"
format_stamp: "Formatted at 2026-08-08 02:58:31 on dist-test-slave-6ntk"
server_key: "ccb14ad9463e36aaea17a8fd6cf01719"
server_key_iv: "0ab95428e7608d4eb61ae7d455da4d07"
server_key_version: "encryptionkey@0"
I20260808 02:58:31.301311 14812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "fded01e6a1a24e79ae5b14a4285ff58a"
format_stamp: "Formatted at 2026-08-08 02:58:31 on dist-test-slave-6ntk"
server_key: "ccb14ad9463e36aaea17a8fd6cf01719"
server_key_iv: "0ab95428e7608d4eb61ae7d455da4d07"
server_key_version: "encryptionkey@0"
I20260808 02:58:31.306631 14812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 02:58:31.310227 14842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:31.311578 14812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 02:58:31.311779 14812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "fded01e6a1a24e79ae5b14a4285ff58a"
format_stamp: "Formatted at 2026-08-08 02:58:31 on dist-test-slave-6ntk"
server_key: "ccb14ad9463e36aaea17a8fd6cf01719"
server_key_iv: "0ab95428e7608d4eb61ae7d455da4d07"
server_key_version: "encryptionkey@0"
I20260808 02:58:31.312023 14812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:31.318164 14812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:31.318962 14812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:31.319324 14812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:31.338152 14812 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:39407
I20260808 02:58:31.338191 14917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:39407 every 8 connection(s)
I20260808 02:58:31.339555 14812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:58:31.339929 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 14812
I20260808 02:58:31.340193 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:58:31.341041 822 external_mini_cluster.cc:1428] Setting key e69b60f36c141c80c03d82d746da3d33
I20260808 02:58:31.344285 14918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:31.351084 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:31.343229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52392 (local address 127.0.205.190:39407)
0808 02:58:31.344386 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:31.344402 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:31.344455 (+ 53us) server_negotiation.cc:407] Connection header received
0808 02:58:31.346811 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:31.346842 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:31.347396 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:31.347846 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:31.348913 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:31.348922 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:31.348938 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:31.348964 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:31.349079 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:31.349101 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:31.349322 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:31.349469 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 02:58:31.349748 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":379,"threads_started":1}
I20260808 02:58:31.357692 14918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a: Bootstrap starting.
I20260808 02:58:31.360811 14918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:31.361810 14918 log.cc:824] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:58:31.364868 14918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a: No bootstrap required, opened a new log
I20260808 02:58:31.372648 14918 raft_consensus.cc:348] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fded01e6a1a24e79ae5b14a4285ff58a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39407 } }
I20260808 02:58:31.373145 14918 raft_consensus.cc:374] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:31.373272 14918 raft_consensus.cc:729] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fded01e6a1a24e79ae5b14a4285ff58a, State: Initialized, Role: FOLLOWER
I20260808 02:58:31.374009 14918 consensus_queue.cc:260] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fded01e6a1a24e79ae5b14a4285ff58a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39407 } }
I20260808 02:58:31.374261 14918 raft_consensus.cc:388] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:58:31.374436 14918 raft_consensus.cc:482] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:58:31.374629 14918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:31.376870 14918 raft_consensus.cc:504] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fded01e6a1a24e79ae5b14a4285ff58a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39407 } }
I20260808 02:58:31.377431 14918 leader_election.cc:302] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fded01e6a1a24e79ae5b14a4285ff58a; no voters:
I20260808 02:58:31.378103 14918 leader_election.cc:288] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:58:31.378237 14927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:58:31.378990 14927 raft_consensus.cc:686] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [term 1 LEADER]: Becoming Leader. State: Replica: fded01e6a1a24e79ae5b14a4285ff58a, State: Running, Role: LEADER
I20260808 02:58:31.379539 14927 consensus_queue.cc:237] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fded01e6a1a24e79ae5b14a4285ff58a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39407 } }
I20260808 02:58:31.380328 14918 sys_catalog.cc:564] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:58:31.382861 14930 sys_catalog.cc:455] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fded01e6a1a24e79ae5b14a4285ff58a. Latest consensus state: current_term: 1 leader_uuid: "fded01e6a1a24e79ae5b14a4285ff58a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fded01e6a1a24e79ae5b14a4285ff58a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39407 } } }
I20260808 02:58:31.383705 14930 sys_catalog.cc:458] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [sys.catalog]: This master's current role is: LEADER
I20260808 02:58:31.383347 14928 sys_catalog.cc:455] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fded01e6a1a24e79ae5b14a4285ff58a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fded01e6a1a24e79ae5b14a4285ff58a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 39407 } } }
I20260808 02:58:31.384223 14928 sys_catalog.cc:458] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a [sys.catalog]: This master's current role is: LEADER
I20260808 02:58:31.385195 14937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:58:31.390762 14937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:58:31.402004 14937 catalog_manager.cc:1347] Generated new cluster ID: 7d84bb613e70420e921c5a3a9375ce50
I20260808 02:58:31.402144 14937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:58:31.424461 14937 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:58:31.424923 14937 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:58:31.435073 14937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fded01e6a1a24e79ae5b14a4285ff58a: Generated new TSK 0
I20260808 02:58:31.435926 14937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:58:31.449363 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39407
--builtin_ntp_servers=127.0.205.148:36489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:58:31.857914 14954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:31.858258 14954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:31.858434 14954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:31.866099 14954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:31.866195 14954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:31.866405 14954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:58:31.875914 14954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:36489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:31.877264 14954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:31.879163 14954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:31.894176 14959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:31.894407 14960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:31.894363 14964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:31.895318 14954 server_base.cc:1034] running on GCE node
I20260808 02:58:31.896011 14954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:31.897065 14954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:31.898310 14954 hybrid_clock.cc:648] HybridClock initialized: now 1786157911898252 us; error 55 us; skew 500 ppm
I20260808 02:58:31.898691 14954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:58:31.901206 14954 webserver.cc:460] Webserver started at http://127.0.205.129:38155/ using document root <none> and password file <none>
I20260808 02:58:31.902036 14954 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:31.902184 14954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:31.902529 14954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:31.905572 14954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "30e8abcb009a46fca9f173ee7b4eb4cf"
format_stamp: "Formatted at 2026-08-08 02:58:31 on dist-test-slave-6ntk"
server_key: "cb815965e00f67c68037d1f7ee0db876"
server_key_iv: "94241d34d3d81e0006da27a7fed232ce"
server_key_version: "encryptionkey@0"
I20260808 02:58:31.906466 14954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "30e8abcb009a46fca9f173ee7b4eb4cf"
format_stamp: "Formatted at 2026-08-08 02:58:31 on dist-test-slave-6ntk"
server_key: "cb815965e00f67c68037d1f7ee0db876"
server_key_iv: "94241d34d3d81e0006da27a7fed232ce"
server_key_version: "encryptionkey@0"
I20260808 02:58:31.911909 14954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:58:31.915659 14971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:31.917264 14954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 02:58:31.917452 14954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "30e8abcb009a46fca9f173ee7b4eb4cf"
format_stamp: "Formatted at 2026-08-08 02:58:31 on dist-test-slave-6ntk"
server_key: "cb815965e00f67c68037d1f7ee0db876"
server_key_iv: "94241d34d3d81e0006da27a7fed232ce"
server_key_version: "encryptionkey@0"
I20260808 02:58:31.917708 14954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:31.923736 14954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:31.924584 14954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:31.924876 14954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:31.925855 14954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:31.927441 14954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:31.927578 14954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:31.927707 14954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:31.927796 14954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:31.961843 14954 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:37855
I20260808 02:58:31.961946 15150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:37855 every 8 connection(s)
I20260808 02:58:31.963310 14954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:58:31.965595 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 14954
I20260808 02:58:31.965839 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:58:31.966758 822 external_mini_cluster.cc:1428] Setting key e1ab734fca254decaa1dfbddc427925c
I20260808 02:58:31.970109 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39407
--builtin_ntp_servers=127.0.205.148:36489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 02:58:31.973122 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:31.966174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:39085 (local address 127.0.205.190:39407)
0808 02:58:31.966703 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:31.966708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:31.968323 (+ 1615us) server_negotiation.cc:407] Connection header received
0808 02:58:31.970456 (+ 2133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:31.970460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:31.970512 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:31.970651 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:31.972218 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:31.972227 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:31.972234 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:31.972259 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:31.972320 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:31.972340 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:31.972494 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:31.972650 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 02:58:31.972725 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":289,"threads_started":1}
I20260808 02:58:31.974022 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:31.966963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:39407 (local address 127.0.205.129:39085)
0808 02:58:31.968130 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:31.968191 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:31.970061 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:31.970923 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:31.970932 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:31.971031 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:31.971779 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:58:31.971782 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:31.971863 (+ 81us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:58:31.971996 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:31.972448 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:31.972456 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:31.972462 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 02:58:31.972685 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 02:58:31.972925 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":340,"threads_started":1}
I20260808 02:58:31.975723 15152 heartbeater.cc:347] Connected to a master server at 127.0.205.190:39407
I20260808 02:58:31.976044 15152 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:31.976677 15152 heartbeater.cc:511] Master 127.0.205.190:39407 requested a full tablet report, sending...
I20260808 02:58:31.978768 14867 ts_manager.cc:194] Registered new tserver with Master: 30e8abcb009a46fca9f173ee7b4eb4cf (127.0.205.129:37855)
W20260808 02:58:32.322588 15157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:32.322886 15157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:32.323051 15157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:32.330587 15157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:32.330703 15157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:32.330889 15157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:58:32.340497 15157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:36489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:32.341853 15157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:32.343794 15157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:32.358086 15172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:32.358325 15168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:32.358403 15169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:32.359062 15157 server_base.cc:1034] running on GCE node
I20260808 02:58:32.359787 15157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:32.360812 15157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:32.362061 15157 hybrid_clock.cc:648] HybridClock initialized: now 1786157912362012 us; error 40 us; skew 500 ppm
I20260808 02:58:32.362407 15157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:58:32.364792 15157 webserver.cc:460] Webserver started at http://127.0.205.130:46739/ using document root <none> and password file <none>
I20260808 02:58:32.365525 15157 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:32.365669 15157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:32.366083 15157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:32.369035 15157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "3637e0d2bd224a15b25c35e364a89026"
format_stamp: "Formatted at 2026-08-08 02:58:32 on dist-test-slave-6ntk"
server_key: "2d943ec2a9fccd396ed8dd03fccaa354"
server_key_iv: "6e354b08655ce4130bddf8feda2e4a7e"
server_key_version: "encryptionkey@0"
I20260808 02:58:32.369974 15157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "3637e0d2bd224a15b25c35e364a89026"
format_stamp: "Formatted at 2026-08-08 02:58:32 on dist-test-slave-6ntk"
server_key: "2d943ec2a9fccd396ed8dd03fccaa354"
server_key_iv: "6e354b08655ce4130bddf8feda2e4a7e"
server_key_version: "encryptionkey@0"
I20260808 02:58:32.375591 15157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 02:58:32.379387 15181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:32.380853 15157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 02:58:32.381040 15157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "3637e0d2bd224a15b25c35e364a89026"
format_stamp: "Formatted at 2026-08-08 02:58:32 on dist-test-slave-6ntk"
server_key: "2d943ec2a9fccd396ed8dd03fccaa354"
server_key_iv: "6e354b08655ce4130bddf8feda2e4a7e"
server_key_version: "encryptionkey@0"
I20260808 02:58:32.381346 15157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:32.387187 15157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:32.388085 15157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:32.388403 15157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:32.389331 15157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:32.390764 15157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:32.390883 15157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:32.391021 15157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:32.391114 15157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:32.428997 15157 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:45809
I20260808 02:58:32.429052 15348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:45809 every 8 connection(s)
I20260808 02:58:32.430553 15157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:58:32.431005 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 15157
I20260808 02:58:32.431216 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:58:32.431864 822 external_mini_cluster.cc:1428] Setting key 07be14e883d6e71344f2f729d6e0897e
I20260808 02:58:32.435117 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39407
--builtin_ntp_servers=127.0.205.148:36489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 02:58:32.439779 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:32.433369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:44397 (local address 127.0.205.190:39407)
0808 02:58:32.433557 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:32.433561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:32.435429 (+ 1868us) server_negotiation.cc:407] Connection header received
0808 02:58:32.437408 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:32.437415 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:32.437485 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:32.437604 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:32.438970 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:32.438978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:32.438985 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:32.439009 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:32.439054 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:32.439073 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:32.439211 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:32.439364 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 02:58:32.439445 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:58:32.440577 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:32.434146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:39407 (local address 127.0.205.130:44397)
0808 02:58:32.435271 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:32.435318 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:32.437127 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:32.437792 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:32.437799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:32.437952 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:32.438661 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:58:32.438664 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:32.438702 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:58:32.438788 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:32.439173 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:32.439181 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:32.439186 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:58:32.439401 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 02:58:32.439591 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":349,"threads_started":1}
I20260808 02:58:32.442073 15349 heartbeater.cc:347] Connected to a master server at 127.0.205.190:39407
I20260808 02:58:32.442395 15349 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:32.442956 15349 heartbeater.cc:511] Master 127.0.205.190:39407 requested a full tablet report, sending...
I20260808 02:58:32.444404 14867 ts_manager.cc:194] Registered new tserver with Master: 3637e0d2bd224a15b25c35e364a89026 (127.0.205.130:45809)
W20260808 02:58:32.784806 15353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:32.785089 15353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:32.785234 15353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:32.793280 15353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:32.793398 15353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:32.793614 15353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:58:32.802812 15353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:36489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:39407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:32.804174 15353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:32.806110 15353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:32.819499 15365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:32.819591 15367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:32.819613 15364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:32.820627 15353 server_base.cc:1034] running on GCE node
I20260808 02:58:32.821297 15353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:32.822410 15353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:32.823622 15353 hybrid_clock.cc:648] HybridClock initialized: now 1786157912823559 us; error 52 us; skew 500 ppm
I20260808 02:58:32.823969 15353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 02:58:32.826467 15353 webserver.cc:460] Webserver started at http://127.0.205.131:36467/ using document root <none> and password file <none>
I20260808 02:58:32.827198 15353 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:32.827335 15353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:32.827665 15353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:32.830700 15353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "f43b23df70d348ccabd3c8787799def5"
format_stamp: "Formatted at 2026-08-08 02:58:32 on dist-test-slave-6ntk"
server_key: "88de0addaced90cc377a1bce20c77335"
server_key_iv: "55397e4f007aad67cebf997395034235"
server_key_version: "encryptionkey@0"
I20260808 02:58:32.831518 15353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "f43b23df70d348ccabd3c8787799def5"
format_stamp: "Formatted at 2026-08-08 02:58:32 on dist-test-slave-6ntk"
server_key: "88de0addaced90cc377a1bce20c77335"
server_key_iv: "55397e4f007aad67cebf997395034235"
server_key_version: "encryptionkey@0"
I20260808 02:58:32.836716 15353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 02:58:32.840226 15377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:32.841564 15353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 02:58:32.841746 15353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "f43b23df70d348ccabd3c8787799def5"
format_stamp: "Formatted at 2026-08-08 02:58:32 on dist-test-slave-6ntk"
server_key: "88de0addaced90cc377a1bce20c77335"
server_key_iv: "55397e4f007aad67cebf997395034235"
server_key_version: "encryptionkey@0"
I20260808 02:58:32.842054 15353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:32.847774 15353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:32.848562 15353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:32.848846 15353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:32.849746 15353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:32.851198 15353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:32.851337 15353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:32.851465 15353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:32.851553 15353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:32.887605 15353 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:35737
I20260808 02:58:32.887651 15539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:35737 every 8 connection(s)
I20260808 02:58:32.889045 15353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:58:32.895587 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 15353
I20260808 02:58:32.895803 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:58:32.896304 822 external_mini_cluster.cc:1428] Setting key a2f420f786c7bae61d5031e40aed591f
I20260808 02:58:32.898406 15154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:32.891808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:59797 (local address 127.0.205.190:39407)
0808 02:58:32.892037 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:32.892043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:32.893494 (+ 1451us) server_negotiation.cc:407] Connection header received
0808 02:58:32.895613 (+ 2119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:32.895622 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:32.895702 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:32.895828 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:32.897522 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:32.897527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:32.897530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:32.897546 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:32.897603 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:32.897619 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:32.897741 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:32.898040 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 02:58:32.898096 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 02:58:32.899855 15542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:32.892326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:39407 (local address 127.0.205.131:59797)
0808 02:58:32.893311 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:32.893366 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:32.895295 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:32.896171 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:32.896185 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:32.896304 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:32.897197 (+ 893us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:58:32.897200 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:32.897240 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:58:32.897333 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:32.897721 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:32.897759 (+ 38us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:32.897769 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 02:58:32.898162 (+ 393us) client_negotiation.cc:240] Negotiation successful
0808 02:58:32.898459 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":300,"threads_started":1}
I20260808 02:58:32.901350 15540 heartbeater.cc:347] Connected to a master server at 127.0.205.190:39407
I20260808 02:58:32.901643 15540 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:32.902307 15540 heartbeater.cc:511] Master 127.0.205.190:39407 requested a full tablet report, sending...
I20260808 02:58:32.903931 14867 ts_manager.cc:194] Registered new tserver with Master: f43b23df70d348ccabd3c8787799def5 (127.0.205.131:35737)
I20260808 02:58:32.912709 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:58:32.919009 15154 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:58:32.917466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37082 (local address 127.0.205.190:39407)
0808 02:58:32.917632 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:32.917636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:32.918177 (+ 541us) server_negotiation.cc:407] Connection header received
0808 02:58:32.918314 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:32.918318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:32.918374 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:32.918457 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:32.918782 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37082: BlockingRecv error: recv got EOF from 127.0.0.1:37082 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 02:58:32.920145 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 14954
I20260808 02:58:32.923933 15551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:32.921198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.0.205.129:37855)
0808 02:58:32.921828 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:32.921835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:32.921926 (+ 91us) server_negotiation.cc:407] Connection header received
0808 02:58:32.922004 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:32.922010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:32.922250 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:32.922377 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:32.923320 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:32.923326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:32.923338 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:32.923356 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:32.923403 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:32.923417 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:32.923523 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:32.923608 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 02:58:32.923689 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":340,"threads_started":1}
I20260808 02:58:32.925179 15145 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:33.005486 15152 heartbeater.cc:503] Master 127.0.205.190:39407 was elected leader, sending a full tablet report...
I20260808 02:58:33.006881 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 14954
W20260808 02:58:33.028549 14843 connection.cc:535] server connection from 127.0.205.129:39085 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:58:33.028873 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 15157
I20260808 02:58:33.032668 15554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:33.029860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.0.205.130:45809)
0808 02:58:33.030423 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:33.030429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:33.030458 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:58:33.030518 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:33.030524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:33.030744 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:33.030928 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:33.031933 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:33.031939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:33.031950 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:33.031962 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:33.032002 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:33.032019 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:33.032172 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:33.032282 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 02:58:33.032413 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":212,"threads_started":1}
I20260808 02:58:33.034068 15342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:33.116345 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 15157
I20260808 02:58:33.140015 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 15353
I20260808 02:58:33.144315 15556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:33.141085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49258 (local address 127.0.205.131:35737)
0808 02:58:33.141744 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:33.141751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:33.141781 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:58:33.141856 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:33.141863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:33.142149 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:33.142325 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:33.143536 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:33.143542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:33.143554 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:58:33.143571 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:33.143610 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:58:33.143624 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:33.143819 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:33.143912 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 02:58:33.144021 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":335,"threads_started":1}
I20260808 02:58:33.145705 15535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:33.232009 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 15353
I20260808 02:58:33.259023 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 14812
I20260808 02:58:33.260349 14904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:33.368193 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 14812
2026-08-08T02:58:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2554 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:58:33 dist-test-slave-6ntk krb5kdc[15564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:58:33 dist-test-slave-6ntk krb5kdc[15564](info): set up 2 sockets
Aug 08 02:58:33 dist-test-slave-6ntk krb5kdc[15564](info): commencing operation
krb5kdc: starting...
W20260808 02:58:34.726199 822 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:58:34 dist-test-slave-6ntk krb5kdc[15564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:58:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:58:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:58:35.033951 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:45343
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:45343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:58:35.391849 15584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:35.392189 15584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:35.392283 15584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:35.399783 15584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:58:35.399919 15584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:35.399968 15584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:35.400019 15584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:58:35.400065 15584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:58:35.410531 15584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:45343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:45343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:35.412019 15584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:35.414121 15584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:35.427017 15593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:35.426950 15596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:35.427544 15591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:35.428367 15584 server_base.cc:1034] running on GCE node
I20260808 02:58:35.429278 15584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:35.430835 15584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:35.432089 15584 hybrid_clock.cc:648] HybridClock initialized: now 1786157915432044 us; error 55 us; skew 500 ppm
I20260808 02:58:35.432472 15584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:35 dist-test-slave-6ntk krb5kdc[15564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157915, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:35.435899 15584 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:58:35.437505 15584 webserver.cc:460] Webserver started at http://127.0.205.190:43929/ using document root <none> and password file <none>
I20260808 02:58:35.438359 15584 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:35.438519 15584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:35.438843 15584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:35.442093 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "6bc235856b234f8aa91f3dadb9fb8940"
format_stamp: "Formatted at 2026-08-08 02:58:35 on dist-test-slave-6ntk"
server_key: "597411be7538b5a386f031c1ea5190d8"
server_key_iv: "81506bce75d220e7d65aa27ac779351d"
server_key_version: "encryptionkey@0"
I20260808 02:58:35.442992 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "6bc235856b234f8aa91f3dadb9fb8940"
format_stamp: "Formatted at 2026-08-08 02:58:35 on dist-test-slave-6ntk"
server_key: "597411be7538b5a386f031c1ea5190d8"
server_key_iv: "81506bce75d220e7d65aa27ac779351d"
server_key_version: "encryptionkey@0"
I20260808 02:58:35.448357 15584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:58:35.451917 15605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:35.453462 15584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:58:35.453663 15584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "6bc235856b234f8aa91f3dadb9fb8940"
format_stamp: "Formatted at 2026-08-08 02:58:35 on dist-test-slave-6ntk"
server_key: "597411be7538b5a386f031c1ea5190d8"
server_key_iv: "81506bce75d220e7d65aa27ac779351d"
server_key_version: "encryptionkey@0"
I20260808 02:58:35.453879 15584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:35.470707 15584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:35.475692 15584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:35.476011 15584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:35.494190 15584 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:45343
I20260808 02:58:35.494217 15683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:45343 every 8 connection(s)
I20260808 02:58:35.495608 15584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:58:35.499558 15684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:35.504235 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 15584
I20260808 02:58:35.504494 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:58:35.505191 822 external_mini_cluster.cc:1428] Setting key 735e3b945f129f89acda1bebc07bbaf2
I20260808 02:58:35.511008 15684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940: Bootstrap starting.
I20260808 02:58:35.515161 15684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:35.516441 15684 log.cc:824] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940: Log is configured to *not* fsync() on all Append() calls
I20260808 02:58:35.520255 15684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940: No bootstrap required, opened a new log
Aug 08 02:58:35 dist-test-slave-6ntk krb5kdc[15564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:58:35.529654 15684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc235856b234f8aa91f3dadb9fb8940" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45343 } }
I20260808 02:58:35.530110 15684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:35.530254 15684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bc235856b234f8aa91f3dadb9fb8940, State: Initialized, Role: FOLLOWER
I20260808 02:58:35.530373 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:35.507314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60118 (local address 127.0.205.190:45343)
0808 02:58:35.508286 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:35.508299 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:35.508344 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:58:35.510117 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:35.510143 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:35.510579 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:35.510914 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:35.512843 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:35.514264 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:35.516052 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:35.516819 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:35.517036 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:35.522370 (+ 5334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:35.522378 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:35.522403 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:35.522444 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:35.525801 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:35.526789 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:35.526796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:35.526802 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:35.526987 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:35.527415 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:35.527418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:35.527420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:35.528018 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:35.528261 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:35.528726 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 02:58:35.529123 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":444,"threads_started":1}
I20260808 02:58:35.531008 15684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc235856b234f8aa91f3dadb9fb8940" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45343 } }
I20260808 02:58:35.531303 15684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:58:35.531451 15684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:58:35.531631 15684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:35.534334 15684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc235856b234f8aa91f3dadb9fb8940" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45343 } }
I20260808 02:58:35.534780 15684 leader_election.cc:302] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bc235856b234f8aa91f3dadb9fb8940; no voters:
I20260808 02:58:35.535365 15684 leader_election.cc:288] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:58:35.535507 15690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:58:35.536293 15690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [term 1 LEADER]: Becoming Leader. State: Replica: 6bc235856b234f8aa91f3dadb9fb8940, State: Running, Role: LEADER
I20260808 02:58:35.536880 15690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc235856b234f8aa91f3dadb9fb8940" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45343 } }
I20260808 02:58:35.537559 15684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:58:35.539656 15691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bc235856b234f8aa91f3dadb9fb8940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc235856b234f8aa91f3dadb9fb8940" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45343 } } }
I20260808 02:58:35.540033 15691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [sys.catalog]: This master's current role is: LEADER
I20260808 02:58:35.541792 15695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:58:35.541697 15692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bc235856b234f8aa91f3dadb9fb8940. Latest consensus state: current_term: 1 leader_uuid: "6bc235856b234f8aa91f3dadb9fb8940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc235856b234f8aa91f3dadb9fb8940" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 45343 } } }
I20260808 02:58:35.542100 15692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940 [sys.catalog]: This master's current role is: LEADER
I20260808 02:58:35.547184 15695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:58:35.558398 15695 catalog_manager.cc:1347] Generated new cluster ID: de019175b0914fb8ba96ca5e7f7605e9
I20260808 02:58:35.558595 15695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:58:35.590888 15695 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:58:35.592849 15695 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:58:35.606186 15695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940: Generated new TSK 0
I20260808 02:58:35.607317 15695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:58:35.682600 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45343
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:58:36.031875 15718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:36.032202 15718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:36.032372 15718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:36.039942 15718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:36.040098 15718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:36.040321 15718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:58:36.050794 15718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:36.052188 15718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:36.053548 15718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:36.067363 15728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:36.067493 15725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:36.067410 15726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:36.068492 15718 server_base.cc:1034] running on GCE node
I20260808 02:58:36.069087 15718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:36.070072 15718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:36.071270 15718 hybrid_clock.cc:648] HybridClock initialized: now 1786157916071208 us; error 54 us; skew 500 ppm
I20260808 02:58:36.071552 15718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:36 dist-test-slave-6ntk krb5kdc[15564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157916, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:36.074813 15718 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:58:36.076370 15718 webserver.cc:460] Webserver started at http://127.0.205.129:39403/ using document root <none> and password file <none>
I20260808 02:58:36.077137 15718 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:36.077284 15718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:36.077621 15718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:36.080830 15718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "04b519b6bf584f82a60e4e3823d9b994"
format_stamp: "Formatted at 2026-08-08 02:58:36 on dist-test-slave-6ntk"
server_key: "69eaf623ac7c1b437369d82e226a9e1a"
server_key_iv: "02e9cc45c37afa52128d59ccb936174d"
server_key_version: "encryptionkey@0"
I20260808 02:58:36.081836 15718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "04b519b6bf584f82a60e4e3823d9b994"
format_stamp: "Formatted at 2026-08-08 02:58:36 on dist-test-slave-6ntk"
server_key: "69eaf623ac7c1b437369d82e226a9e1a"
server_key_iv: "02e9cc45c37afa52128d59ccb936174d"
server_key_version: "encryptionkey@0"
I20260808 02:58:36.087612 15718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 02:58:36.091511 15735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:36.093075 15718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 02:58:36.093279 15718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "04b519b6bf584f82a60e4e3823d9b994"
format_stamp: "Formatted at 2026-08-08 02:58:36 on dist-test-slave-6ntk"
server_key: "69eaf623ac7c1b437369d82e226a9e1a"
server_key_iv: "02e9cc45c37afa52128d59ccb936174d"
server_key_version: "encryptionkey@0"
I20260808 02:58:36.093520 15718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:36.105844 15718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:36.110605 15718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:36.110915 15718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:36.111886 15718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:36.113324 15718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:36.113396 15718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:36.113567 15718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:36.113633 15718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:36.146974 15718 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:41023
I20260808 02:58:36.147054 15920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:41023 every 8 connection(s)
I20260808 02:58:36.148587 15718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:58:36.153338 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 15718
I20260808 02:58:36.153574 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:58:36.154286 822 external_mini_cluster.cc:1428] Setting key 43c0dc09865631695943f2040840b430
Aug 08 02:58:36 dist-test-slave-6ntk krb5kdc[15564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157916, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:58:36.172462 15924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:36.151421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:60269 (local address 127.0.205.190:45343)
0808 02:58:36.151879 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:36.151884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:36.152868 (+ 984us) server_negotiation.cc:407] Connection header received
0808 02:58:36.155088 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:36.155096 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:36.155183 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:36.155387 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:36.157371 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:36.158602 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:36.160247 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:36.160942 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:36.161092 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:36.164884 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:36.164894 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:36.164902 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:36.164952 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:36.168821 (+ 3869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:36.169957 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:36.169961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:36.169964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:36.170088 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:36.170635 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:36.170638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:36.170640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:36.170936 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:36.171057 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:36.172021 (+ 964us) server_negotiation.cc:299] Negotiation successful
0808 02:58:36.172197 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":229,"threads_started":1}
I20260808 02:58:36.174178 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:36.151962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:45343 (local address 127.0.205.129:60269)
0808 02:58:36.152695 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:36.152742 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:36.154724 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:36.155658 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:36.155669 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:36.156505 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:36.157170 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:36.157189 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:36.158784 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:36.158791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:36.160083 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:36.160096 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:36.161159 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:36.161164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:36.161370 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:36.162133 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:36.162156 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:36.164641 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:36.168996 (+ 4355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:36.169007 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:36.169030 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:36.169767 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:36.170204 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:36.170208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:36.170212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:36.170514 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:36.171084 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:36.171095 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:36.171615 (+ 520us) client_negotiation.cc:769] Sending connection context
0808 02:58:36.171944 (+ 329us) client_negotiation.cc:240] Negotiation successful
0808 02:58:36.172576 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":283,"threads_started":1}
I20260808 02:58:36.175966 15921 heartbeater.cc:347] Connected to a master server at 127.0.205.190:45343
I20260808 02:58:36.176365 15921 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:36.177134 15921 heartbeater.cc:511] Master 127.0.205.190:45343 requested a full tablet report, sending...
I20260808 02:58:36.179557 15636 ts_manager.cc:194] Registered new tserver with Master: 04b519b6bf584f82a60e4e3823d9b994 (127.0.205.129:41023)
I20260808 02:58:36.181084 15636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:60269
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 02:58:36.226478 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45343
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:58:36.577989 15932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:36.578378 15932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:36.578542 15932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:36.586279 15932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:36.586459 15932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:36.586633 15932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:58:36.596984 15932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:36.598534 15932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:36.599897 15932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:58:36.613530 15941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:36.613423 15942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:36.613493 15944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:36.614599 15932 server_base.cc:1034] running on GCE node
I20260808 02:58:36.615300 15932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:36.616283 15932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:36.617515 15932 hybrid_clock.cc:648] HybridClock initialized: now 1786157916617451 us; error 60 us; skew 500 ppm
I20260808 02:58:36.617857 15932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:36 dist-test-slave-6ntk krb5kdc[15564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157916, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:36.624934 15932 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:58:36.626552 15932 webserver.cc:460] Webserver started at http://127.0.205.130:43601/ using document root <none> and password file <none>
I20260808 02:58:36.627300 15932 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:36.627456 15932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:36.627776 15932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:36.630918 15932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "fd78d40344ac4ccbbca76de919fdedef"
format_stamp: "Formatted at 2026-08-08 02:58:36 on dist-test-slave-6ntk"
server_key: "789c824fce39f1bfcf787d09ad663978"
server_key_iv: "9b221467f6cf91cd23d8834fbb79c0e7"
server_key_version: "encryptionkey@0"
I20260808 02:58:36.631723 15932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "fd78d40344ac4ccbbca76de919fdedef"
format_stamp: "Formatted at 2026-08-08 02:58:36 on dist-test-slave-6ntk"
server_key: "789c824fce39f1bfcf787d09ad663978"
server_key_iv: "9b221467f6cf91cd23d8834fbb79c0e7"
server_key_version: "encryptionkey@0"
I20260808 02:58:36.637187 15932 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260808 02:58:36.641260 15958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:36.643042 15932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 02:58:36.643262 15932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "fd78d40344ac4ccbbca76de919fdedef"
format_stamp: "Formatted at 2026-08-08 02:58:36 on dist-test-slave-6ntk"
server_key: "789c824fce39f1bfcf787d09ad663978"
server_key_iv: "9b221467f6cf91cd23d8834fbb79c0e7"
server_key_version: "encryptionkey@0"
I20260808 02:58:36.643548 15932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:36.671049 15932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:36.675962 15932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:36.676298 15932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:36.677421 15932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:36.678877 15932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:36.678992 15932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:36.679155 15932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:36.679232 15932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:36.714956 15932 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:37093
I20260808 02:58:36.715008 16122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:37093 every 8 connection(s)
I20260808 02:58:36.716332 15932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:58:36.716929 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 15932
I20260808 02:58:36.717166 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:58:36.717739 822 external_mini_cluster.cc:1428] Setting key 52b6a865e413db95e5525723874c1352
Aug 08 02:58:36 dist-test-slave-6ntk krb5kdc[15564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157916, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:58:36.738755 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:36.719151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:34627 (local address 127.0.205.190:45343)
0808 02:58:36.719637 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:36.719641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:36.720954 (+ 1313us) server_negotiation.cc:407] Connection header received
0808 02:58:36.722904 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:36.722913 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:36.723004 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:36.723127 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:36.724640 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:36.725782 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:36.727477 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:36.728234 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:36.728395 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:36.732404 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:36.732409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:36.732412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:36.732444 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:36.735094 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:36.736092 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:36.736096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:36.736098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:36.736230 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:36.736773 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:36.736776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:36.736778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:36.737048 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:36.737160 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:36.738194 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0808 02:58:36.738380 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":281,"threads_started":1}
I20260808 02:58:36.740352 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:36.719890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:45343 (local address 127.0.205.130:34627)
0808 02:58:36.720778 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:36.720826 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:36.722634 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:36.723324 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:36.723352 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:36.723867 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:36.724458 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:36.724476 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:36.726011 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:36.726015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:36.727304 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:36.727317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:36.728465 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:36.728469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:36.728673 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:36.729456 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:36.729481 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:36.732124 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:36.735243 (+ 3119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:36.735255 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:36.735276 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:36.735971 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:36.736343 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:36.736350 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:36.736356 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:36.736655 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:36.737180 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:36.737190 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:36.737718 (+ 528us) client_negotiation.cc:769] Sending connection context
0808 02:58:36.738105 (+ 387us) client_negotiation.cc:240] Negotiation successful
0808 02:58:36.738760 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":377,"threads_started":1}
I20260808 02:58:36.742035 16123 heartbeater.cc:347] Connected to a master server at 127.0.205.190:45343
I20260808 02:58:36.742381 16123 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:36.743150 16123 heartbeater.cc:511] Master 127.0.205.190:45343 requested a full tablet report, sending...
I20260808 02:58:36.744704 15636 ts_manager.cc:194] Registered new tserver with Master: fd78d40344ac4ccbbca76de919fdedef (127.0.205.130:37093)
I20260808 02:58:36.745610 15636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:34627
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:58:36.789235 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45343
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:58:37.152406 16134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:58:37.152727 16134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:58:37.152897 16134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:58:37.160403 16134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:58:37.160581 16134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:58:37.160763 16134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:58:37.170809 16134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:45343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:58:37.172312 16134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:58:37.173717 16134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:58:37.184479 15921 heartbeater.cc:503] Master 127.0.205.190:45343 was elected leader, sending a full tablet report...
W20260808 02:58:37.188520 16145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:37.188546 16150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:58:37.189069 16148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:58:37.189690 16134 server_base.cc:1034] running on GCE node
I20260808 02:58:37.190368 16134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:58:37.191311 16134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:58:37.192550 16134 hybrid_clock.cc:648] HybridClock initialized: now 1786157917192489 us; error 53 us; skew 500 ppm
I20260808 02:58:37.192896 16134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:58:37 dist-test-slave-6ntk krb5kdc[15564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157917, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:58:37.196270 16134 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:58:37.197989 16134 webserver.cc:460] Webserver started at http://127.0.205.131:45027/ using document root <none> and password file <none>
I20260808 02:58:37.198711 16134 fs_manager.cc:359] Metadata directory not provided
I20260808 02:58:37.198815 16134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:58:37.199071 16134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:58:37.202039 16134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "591e16444cee4cdfad7257be5be47e2c"
format_stamp: "Formatted at 2026-08-08 02:58:37 on dist-test-slave-6ntk"
server_key: "e8e02b5ec34f9578127e2b0f81286824"
server_key_iv: "4e83d991b52f2bf69914c738ee98bfd2"
server_key_version: "encryptionkey@0"
I20260808 02:58:37.202776 16134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "591e16444cee4cdfad7257be5be47e2c"
format_stamp: "Formatted at 2026-08-08 02:58:37 on dist-test-slave-6ntk"
server_key: "e8e02b5ec34f9578127e2b0f81286824"
server_key_iv: "4e83d991b52f2bf69914c738ee98bfd2"
server_key_version: "encryptionkey@0"
I20260808 02:58:37.207993 16134 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 02:58:37.211786 16160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:37.213224 16134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 02:58:37.213430 16134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "591e16444cee4cdfad7257be5be47e2c"
format_stamp: "Formatted at 2026-08-08 02:58:37 on dist-test-slave-6ntk"
server_key: "e8e02b5ec34f9578127e2b0f81286824"
server_key_iv: "4e83d991b52f2bf69914c738ee98bfd2"
server_key_version: "encryptionkey@0"
I20260808 02:58:37.213660 16134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:58:37.225831 16134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:58:37.230422 16134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:58:37.230726 16134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:58:37.231747 16134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:58:37.233103 16134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:58:37.233208 16134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:37.233362 16134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:58:37.233435 16134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:58:37.268122 16134 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:35487
I20260808 02:58:37.268158 16339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:35487 every 8 connection(s)
I20260808 02:58:37.269488 16134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:58:37.279592 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 16134
I20260808 02:58:37.279829 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:58:37.280280 822 external_mini_cluster.cc:1428] Setting key c2ca0174e965bf5238540125ab02420e
Aug 08 02:58:37 dist-test-slave-6ntk krb5kdc[15564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157917, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:58:37.291747 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.272232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:45433 (local address 127.0.205.190:45343)
0808 02:58:37.272732 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.272737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.273741 (+ 1004us) server_negotiation.cc:407] Connection header received
0808 02:58:37.275715 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.275721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.275833 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.275959 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:37.277652 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.278616 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.280987 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.281564 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.281734 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.286327 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:37.286333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:37.286336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:37.286371 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:37.288948 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:37.289635 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:37.289639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:37.289641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:37.289778 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:37.290223 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:37.290226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:37.290228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:37.290508 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:37.290630 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.291285 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.291453 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":242,"threads_started":1}
I20260808 02:58:37.292774 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.272854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:45343 (local address 127.0.205.131:45433)
0808 02:58:37.273574 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:37.273613 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:37.275423 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:37.276219 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:37.276229 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:37.276759 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:58:37.277366 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.277444 (+ 78us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.278849 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.278853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.280794 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.280814 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.281932 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.281940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.282170 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.283133 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:58:37.283167 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:58:37.286105 (+ 2938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:58:37.289077 (+ 2972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:37.289083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:37.289097 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:37.289534 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:37.289925 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:58:37.289929 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:58:37.289931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:58:37.290119 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:58:37.290623 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:58:37.290629 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:58:37.290948 (+ 319us) client_negotiation.cc:769] Sending connection context
0808 02:58:37.291179 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 02:58:37.291582 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":284,"threads_started":1}
I20260808 02:58:37.294309 16341 heartbeater.cc:347] Connected to a master server at 127.0.205.190:45343
I20260808 02:58:37.294699 16341 heartbeater.cc:464] Registering TS with master...
I20260808 02:58:37.295629 16341 heartbeater.cc:511] Master 127.0.205.190:45343 requested a full tablet report, sending...
I20260808 02:58:37.297356 15636 ts_manager.cc:194] Registered new tserver with Master: 591e16444cee4cdfad7257be5be47e2c (127.0.205.131:35487)
I20260808 02:58:37.298346 15636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:45433
I20260808 02:58:37.308984 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:58:37.326800 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.313769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.0.205.190:45343)
0808 02:58:37.314037 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.314042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.314523 (+ 481us) server_negotiation.cc:407] Connection header received
0808 02:58:37.314660 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.314664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.314740 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.314826 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:37.316291 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.317021 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.318530 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.318927 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.319021 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.321386 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:37.321392 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:37.321395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:37.321428 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:37.324188 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:37.324805 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:37.324808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:37.324811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:37.324939 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:37.325282 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:37.325285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:37.325287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:37.325616 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:37.325796 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.326332 (+ 536us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.326524 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 02:58:37.331552 15636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:58:37.338676 15636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:58:37.369701 16366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.352634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38850 (local address 127.0.205.130:37093)
0808 02:58:37.358468 (+ 5834us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.358481 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.358521 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:58:37.358674 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.358686 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.359029 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.359278 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:37.360960 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.363177 (+ 2217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.366857 (+ 3680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.368213 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.368375 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.368575 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.369083 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.369295 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5597,"thread_start_us":5446,"threads_started":1}
I20260808 02:58:37.371088 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.352295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:37093 (local address 127.0.0.1:38850)
0808 02:58:37.353956 (+ 1661us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:37.354008 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:37.354248 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:37.359686 (+ 5438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:37.359698 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:37.360016 (+ 318us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:37.360649 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.360670 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.363422 (+ 2752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.363433 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.366659 (+ 3226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.366671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.368459 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.368468 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.368766 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.368849 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 02:58:37.369019 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 02:58:37.369318 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1308,"thread_start_us":538,"threads_started":1}
I20260808 02:58:37.375802 16367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.352646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:35487 (local address 127.0.0.1:48068)
0808 02:58:37.366150 (+ 13504us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:37.366175 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:37.366331 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:37.367020 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:37.367025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:37.367050 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:37.367482 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.367499 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.369335 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.369340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.371541 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.371561 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.374941 (+ 3380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.374957 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.375146 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.375207 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 02:58:37.375312 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 02:58:37.375458 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13300,"thread_start_us":13004,"threads_started":1}
I20260808 02:58:37.376292 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.353320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.0.205.131:35487)
0808 02:58:37.359471 (+ 6151us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.359480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.366457 (+ 6977us) server_negotiation.cc:407] Connection header received
0808 02:58:37.366562 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.366570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.366817 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.367047 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:37.367710 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.369144 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.371746 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.374659 (+ 2913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.374833 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.375078 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.375391 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.375563 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5966,"thread_start_us":5808,"threads_started":1}
I20260808 02:58:37.375903 16027 tablet_service.cc:1467] Processing CreateTablet for tablet 55bee81b21d545098a6e8ce625c50f3f (DEFAULT_TABLE table=test-table [id=26cf1518c74e4093b7d8dd377c3d353a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:37.378912 16027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55bee81b21d545098a6e8ce625c50f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:37.381330 16371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.359937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52338 (local address 127.0.205.129:41023)
0808 02:58:37.373687 (+ 13750us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.373694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.373718 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:58:37.373810 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.373815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.374062 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.374278 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:37.374869 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.376413 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.378573 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.380079 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.380240 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.380484 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.380745 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.380946 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13591,"thread_start_us":14543,"threads_started":1}
I20260808 02:58:37.381340 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.359222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:41023 (local address 127.0.0.1:52338)
0808 02:58:37.366420 (+ 7198us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:37.366438 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:37.366562 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:37.374256 (+ 7694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:37.374261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:37.374290 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:37.374651 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.374664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.376595 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.376600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.378385 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.378400 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.380341 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.380348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.380535 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.380570 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:58:37.380676 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 02:58:37.380833 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7077,"thread_start_us":6871,"threads_started":1}
I20260808 02:58:37.380506 16234 tablet_service.cc:1467] Processing CreateTablet for tablet 55bee81b21d545098a6e8ce625c50f3f (DEFAULT_TABLE table=test-table [id=26cf1518c74e4093b7d8dd377c3d353a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:37.383097 16234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55bee81b21d545098a6e8ce625c50f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:37.386127 15821 tablet_service.cc:1467] Processing CreateTablet for tablet 55bee81b21d545098a6e8ce625c50f3f (DEFAULT_TABLE table=test-table [id=26cf1518c74e4093b7d8dd377c3d353a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:58:37.387727 15821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55bee81b21d545098a6e8ce625c50f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:58:37.391841 16372 tablet_bootstrap.cc:492] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: Bootstrap starting.
I20260808 02:58:37.396183 16372 tablet_bootstrap.cc:654] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:37.397421 16372 log.cc:824] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: Log is configured to *not* fsync() on all Append() calls
I20260808 02:58:37.399099 16374 tablet_bootstrap.cc:492] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: Bootstrap starting.
I20260808 02:58:37.401146 16372 tablet_bootstrap.cc:492] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: No bootstrap required, opened a new log
I20260808 02:58:37.401458 16372 ts_tablet_manager.cc:1397] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260808 02:58:37.402983 16376 tablet_bootstrap.cc:492] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: Bootstrap starting.
I20260808 02:58:37.403061 16374 tablet_bootstrap.cc:654] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:37.404567 16374 log.cc:824] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:58:37.406703 16376 tablet_bootstrap.cc:654] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: Neither blocks nor log segments found. Creating new log.
I20260808 02:58:37.407784 16376 log.cc:824] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: Log is configured to *not* fsync() on all Append() calls
I20260808 02:58:37.408216 16374 tablet_bootstrap.cc:492] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: No bootstrap required, opened a new log
I20260808 02:58:37.408562 16374 ts_tablet_manager.cc:1397] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260808 02:58:37.410884 16376 tablet_bootstrap.cc:492] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: No bootstrap required, opened a new log
I20260808 02:58:37.411214 16376 ts_tablet_manager.cc:1397] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260808 02:58:37.413489 16372 raft_consensus.cc:348] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 } } peers { permanent_uuid: "591e16444cee4cdfad7257be5be47e2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35487 } } peers { permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 } }
I20260808 02:58:37.413985 16372 raft_consensus.cc:374] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:37.414131 16372 raft_consensus.cc:729] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd78d40344ac4ccbbca76de919fdedef, State: Initialized, Role: FOLLOWER
I20260808 02:58:37.414803 16372 consensus_queue.cc:260] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 } } peers { permanent_uuid: "591e16444cee4cdfad7257be5be47e2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35487 } } peers { permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 } }
I20260808 02:58:37.416028 16123 heartbeater.cc:503] Master 127.0.205.190:45343 was elected leader, sending a full tablet report...
I20260808 02:58:37.416692 16372 ts_tablet_manager.cc:1428] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260808 02:58:37.419310 16374 raft_consensus.cc:348] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 } } peers { permanent_uuid: "591e16444cee4cdfad7257be5be47e2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35487 } } peers { permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 } }
I20260808 02:58:37.419699 16374 raft_consensus.cc:374] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:37.419831 16374 raft_consensus.cc:729] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e16444cee4cdfad7257be5be47e2c, State: Initialized, Role: FOLLOWER
I20260808 02:58:37.420992 16374 consensus_queue.cc:260] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 } } peers { permanent_uuid: "591e16444cee4cdfad7257be5be47e2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35487 } } peers { permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 } }
I20260808 02:58:37.423125 16376 raft_consensus.cc:348] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 } } peers { permanent_uuid: "591e16444cee4cdfad7257be5be47e2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35487 } } peers { permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 } }
I20260808 02:58:37.423650 16376 raft_consensus.cc:374] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:58:37.423661 16341 heartbeater.cc:503] Master 127.0.205.190:45343 was elected leader, sending a full tablet report...
I20260808 02:58:37.423815 16376 raft_consensus.cc:729] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04b519b6bf584f82a60e4e3823d9b994, State: Initialized, Role: FOLLOWER
I20260808 02:58:37.424444 16374 ts_tablet_manager.cc:1428] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260808 02:58:37.424585 16376 consensus_queue.cc:260] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 } } peers { permanent_uuid: "591e16444cee4cdfad7257be5be47e2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35487 } } peers { permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 } }
I20260808 02:58:37.425585 16382 raft_consensus.cc:482] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:58:37.425933 16382 raft_consensus.cc:504] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 } } peers { permanent_uuid: "591e16444cee4cdfad7257be5be47e2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35487 } } peers { permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 } }
I20260808 02:58:37.426507 16376 ts_tablet_manager.cc:1428] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260808 02:58:37.428025 16382 leader_election.cc:288] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd78d40344ac4ccbbca76de919fdedef (127.0.205.130:37093), 04b519b6bf584f82a60e4e3823d9b994 (127.0.205.129:41023)
I20260808 02:58:37.435245 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.428225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:37093 (local address 127.0.205.131:44773)
0808 02:58:37.428406 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:37.428427 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:37.428541 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:37.429476 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:37.429483 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:37.429513 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:37.429732 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.429745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.431204 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.431210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.432705 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.432717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.434474 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.434480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.434680 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.434725 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 02:58:37.434821 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:58:37.434982 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 02:58:37.435302 16366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.428951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:44773 (local address 127.0.205.130:37093)
0808 02:58:37.429149 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.429155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.429175 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:58:37.429242 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.429247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.429319 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.429453 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:37.429962 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.431038 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.432870 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.434143 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.434292 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.434371 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.434886 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.435028 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:58:37.435612 16371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.428999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:60825 (local address 127.0.205.129:41023)
0808 02:58:37.429190 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.429195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.429245 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:58:37.429442 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.429447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.429513 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.429599 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:58:37.430471 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.431448 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.433381 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.434631 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.434796 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.434869 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.435236 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.435387 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 02:58:37.435689 16386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.428611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:41023 (local address 127.0.205.131:60825)
0808 02:58:37.429164 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 02:58:37.429181 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:58:37.429303 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:58:37.429756 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:58:37.429760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:58:37.429785 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:58:37.430264 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.430275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.431611 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.431615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.433201 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:58:37.433216 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.434863 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.434870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:58:37.435050 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.435083 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:58:37.435193 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 02:58:37.435323 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":1245,"threads_started":1}
I20260808 02:58:37.436631 16055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55bee81b21d545098a6e8ce625c50f3f" candidate_uuid: "591e16444cee4cdfad7257be5be47e2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd78d40344ac4ccbbca76de919fdedef" is_pre_election: true
I20260808 02:58:37.436662 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55bee81b21d545098a6e8ce625c50f3f" candidate_uuid: "591e16444cee4cdfad7257be5be47e2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04b519b6bf584f82a60e4e3823d9b994" is_pre_election: true
I20260808 02:58:37.437155 16055 raft_consensus.cc:2454] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 591e16444cee4cdfad7257be5be47e2c in term 0.
I20260808 02:58:37.437166 15857 raft_consensus.cc:2454] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 591e16444cee4cdfad7257be5be47e2c in term 0.
I20260808 02:58:37.437832 16163 leader_election.cc:302] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04b519b6bf584f82a60e4e3823d9b994, 591e16444cee4cdfad7257be5be47e2c; no voters:
I20260808 02:58:37.438295 16382 raft_consensus.cc:2781] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:58:37.438509 16382 raft_consensus.cc:482] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:58:37.438627 16382 raft_consensus.cc:3037] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:37.441022 16382 raft_consensus.cc:504] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 } } peers { permanent_uuid: "591e16444cee4cdfad7257be5be47e2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35487 } } peers { permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 } }
I20260808 02:58:37.441697 16382 leader_election.cc:288] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [CANDIDATE]: Term 1 election: Requested vote from peers fd78d40344ac4ccbbca76de919fdedef (127.0.205.130:37093), 04b519b6bf584f82a60e4e3823d9b994 (127.0.205.129:41023)
I20260808 02:58:37.442301 16055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55bee81b21d545098a6e8ce625c50f3f" candidate_uuid: "591e16444cee4cdfad7257be5be47e2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd78d40344ac4ccbbca76de919fdedef"
I20260808 02:58:37.442415 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55bee81b21d545098a6e8ce625c50f3f" candidate_uuid: "591e16444cee4cdfad7257be5be47e2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04b519b6bf584f82a60e4e3823d9b994"
I20260808 02:58:37.442518 16055 raft_consensus.cc:3037] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:37.442665 15857 raft_consensus.cc:3037] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:58:37.444888 16055 raft_consensus.cc:2454] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e16444cee4cdfad7257be5be47e2c in term 1.
I20260808 02:58:37.444905 15857 raft_consensus.cc:2454] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 591e16444cee4cdfad7257be5be47e2c in term 1.
I20260808 02:58:37.445298 16163 leader_election.cc:302] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04b519b6bf584f82a60e4e3823d9b994, 591e16444cee4cdfad7257be5be47e2c; no voters:
I20260808 02:58:37.445607 16382 raft_consensus.cc:2781] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:58:37.446080 16382 raft_consensus.cc:686] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 1 LEADER]: Becoming Leader. State: Replica: 591e16444cee4cdfad7257be5be47e2c, State: Running, Role: LEADER
I20260808 02:58:37.446589 16382 consensus_queue.cc:237] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 } } peers { permanent_uuid: "591e16444cee4cdfad7257be5be47e2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35487 } } peers { permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 } }
I20260808 02:58:37.451895 15635 catalog_manager.cc:5495] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 591e16444cee4cdfad7257be5be47e2c (127.0.205.131). New cstate: current_term: 1 leader_uuid: "591e16444cee4cdfad7257be5be47e2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "591e16444cee4cdfad7257be5be47e2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 35487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 } health_report { overall_health: UNKNOWN } } }
W20260808 02:58:37.468581 16124 tablet.cc:2369] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:58:37.491935 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.486211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.0.205.131:35487)
0808 02:58:37.486416 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.486422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.486561 (+ 139us) server_negotiation.cc:407] Connection header received
0808 02:58:37.486834 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.486838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.486935 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.487119 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:58:37.487521 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.488271 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.489650 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.490133 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.490271 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.490802 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:58:37.491341 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:58:37.491512 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.491571 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.491657 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 02:58:37.500634 16055 raft_consensus.cc:1261] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef [term 1 FOLLOWER]: Refusing update from remote peer 591e16444cee4cdfad7257be5be47e2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:37.500659 15857 raft_consensus.cc:1261] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994 [term 1 FOLLOWER]: Refusing update from remote peer 591e16444cee4cdfad7257be5be47e2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:58:37.501694 16387 consensus_queue.cc:1035] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "04b519b6bf584f82a60e4e3823d9b994" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:37.502172 16382 consensus_queue.cc:1035] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd78d40344ac4ccbbca76de919fdedef" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 37093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:58:37.519029 16392 mvcc.cc:205] Tried to move back new op lower bound from 7316102830071951360 to 7316102829865607168. Current Snapshot: MvccSnapshot[applied={T|T < 7316102830071951360}]
I20260808 02:58:37.527674 16394 mvcc.cc:205] Tried to move back new op lower bound from 7316102830071951360 to 7316102829865607168. Current Snapshot: MvccSnapshot[applied={T|T < 7316102830071951360}]
I20260808 02:58:37.540030 15635 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:58:37.540401 15635 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:58:37.544232 15635 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bc235856b234f8aa91f3dadb9fb8940: Sending DeleteTablet for 3 replicas of tablet 55bee81b21d545098a6e8ce625c50f3f
I20260808 02:58:37.545610 16234 tablet_service.cc:1514] Processing DeleteTablet for tablet 55bee81b21d545098a6e8ce625c50f3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:37 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:48068
I20260808 02:58:37.545701 16027 tablet_service.cc:1514] Processing DeleteTablet for tablet 55bee81b21d545098a6e8ce625c50f3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:37 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:38850
I20260808 02:58:37.545965 15821 tablet_service.cc:1514] Processing DeleteTablet for tablet 55bee81b21d545098a6e8ce625c50f3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:58:37 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:52338
I20260808 02:58:37.548485 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 15718
I20260808 02:58:37.557727 16402 tablet_replica.cc:330] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: stopping tablet replica
I20260808 02:58:37.558704 16404 tablet_replica.cc:330] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: stopping tablet replica
I20260808 02:58:37.559217 16404 raft_consensus.cc:2229] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:58:37.559880 16404 raft_consensus.cc:2258] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:37.560496 16402 raft_consensus.cc:2229] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:58:37.561091 16402 raft_consensus.cc:2258] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:37.563573 16404 ts_tablet_manager.cc:1905] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:37.570863 16401 tablet_replica.cc:330] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: stopping tablet replica
I20260808 02:58:37.571419 16401 raft_consensus.cc:2229] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:58:37.571695 16402 ts_tablet_manager.cc:1905] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:37.572300 16401 raft_consensus.cc:2258] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:58:37.573812 16404 ts_tablet_manager.cc:1918] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:58:37.574105 16404 log.cc:1196] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/55bee81b21d545098a6e8ce625c50f3f
I20260808 02:58:37.574733 16401 ts_tablet_manager.cc:1905] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:58:37.574843 16404 ts_tablet_manager.cc:1939] T 55bee81b21d545098a6e8ce625c50f3f P 04b519b6bf584f82a60e4e3823d9b994: Deleting consensus metadata
I20260808 02:58:37.576967 15610 catalog_manager.cc:4841] TS 04b519b6bf584f82a60e4e3823d9b994 (127.0.205.129:41023): tablet 55bee81b21d545098a6e8ce625c50f3f (table test-table [id=26cf1518c74e4093b7d8dd377c3d353a]) successfully deleted
I20260808 02:58:37.579476 16402 ts_tablet_manager.cc:1918] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:58:37.579766 16402 log.cc:1196] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/55bee81b21d545098a6e8ce625c50f3f
I20260808 02:58:37.580469 16402 ts_tablet_manager.cc:1939] T 55bee81b21d545098a6e8ce625c50f3f P fd78d40344ac4ccbbca76de919fdedef: Deleting consensus metadata
Aug 08 02:58:37 dist-test-slave-6ntk krb5kdc[15564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:58:37.582724 15610 catalog_manager.cc:4841] TS fd78d40344ac4ccbbca76de919fdedef (127.0.205.130:37093): tablet 55bee81b21d545098a6e8ce625c50f3f (table test-table [id=26cf1518c74e4093b7d8dd377c3d353a]) successfully deleted
I20260808 02:58:37.582791 16401 ts_tablet_manager.cc:1918] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:58:37.582988 16401 log.cc:1196] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/55bee81b21d545098a6e8ce625c50f3f
I20260808 02:58:37.583600 16401 ts_tablet_manager.cc:1939] T 55bee81b21d545098a6e8ce625c50f3f P 591e16444cee4cdfad7257be5be47e2c: Deleting consensus metadata
I20260808 02:58:37.585999 15608 catalog_manager.cc:4841] TS 591e16444cee4cdfad7257be5be47e2c (127.0.205.131:35487): tablet 55bee81b21d545098a6e8ce625c50f3f (table test-table [id=26cf1518c74e4093b7d8dd377c3d353a]) successfully deleted
I20260808 02:58:37.590421 16371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.550377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52340 (local address 127.0.205.129:41023)
0808 02:58:37.550614 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.550622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.570506 (+ 19884us) server_negotiation.cc:407] Connection header received
0808 02:58:37.570874 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.570880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.570994 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.571140 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:37.573105 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.574344 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.576284 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.576975 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.577082 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.583452 (+ 6370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:37.583469 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:37.583483 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:37.583522 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:37.587209 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:37.588010 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:37.588019 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:37.588022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:37.588178 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:37.588619 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:37.588625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:37.588628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:37.589051 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:37.589389 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.589748 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.590029 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:58:37.591181 15913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:37.684144 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 15718
I20260808 02:58:37.712142 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 15932
Aug 08 02:58:37 dist-test-slave-6ntk krb5kdc[15564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:58:37.732657 16366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.713074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.0.205.130:37093)
0808 02:58:37.713318 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.713325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.713349 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:58:37.713429 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.713434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.713532 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.713654 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:37.715330 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.716620 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.718690 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.719344 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.719519 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.724947 (+ 5428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:37.724962 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:37.724983 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:37.725037 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:37.729286 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:37.730177 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:37.730186 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:37.730191 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:37.730368 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:37.730826 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:37.730842 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:37.730845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:37.731373 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:37.731700 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.732059 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.732288 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260808 02:58:37.733433 16116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:37.827098 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 15932
I20260808 02:58:37.850622 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 16134
Aug 08 02:58:37 dist-test-slave-6ntk krb5kdc[15564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:58:37.870180 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:58:37.851586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.0.205.131:35487)
0808 02:58:37.851758 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:58:37.851762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:58:37.851784 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:58:37.851938 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:58:37.851941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:58:37.852007 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:58:37.852090 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:58:37.854221 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.855096 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.857001 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:58:37.857545 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:58:37.857648 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:58:37.863537 (+ 5889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:58:37.863554 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:58:37.863571 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:58:37.863618 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:58:37.866899 (+ 3281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:37.867704 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:37.867711 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:37.867714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:37.867892 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:58:37.868300 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:58:37.868307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:58:37.868310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:58:37.868704 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:58:37.869052 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0808 02:58:37.869361 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 02:58:37.869608 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 02:58:37.871027 16334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:37.998914 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 16134
I20260808 02:58:38.026502 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 15584
I20260808 02:58:38.027712 15679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:58:38.141913 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 15584
2026-08-08T02:58:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4782 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:58:38 dist-test-slave-6ntk krb5kdc[16413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:58:38 dist-test-slave-6ntk krb5kdc[16413](info): set up 2 sockets
Aug 08 02:58:38 dist-test-slave-6ntk krb5kdc[16413](info): commencing operation
krb5kdc: starting...
W20260808 02:58:39.491888 822 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:58:39 dist-test-slave-6ntk krb5kdc[16413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:58:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:58:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.205.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.205.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/rangeradmin_127.0.205.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/rangeradmin_127.0.205.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.205.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.205.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/rangerlookup_127.0.205.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/rangerlookup_127.0.205.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.205.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.205.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/HTTP_127.0.205.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/HTTP_127.0.205.148@KRBTEST.COM.keytab.
I20260808 02:58:39.921087 822 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvzvYzW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/postgres -l logfile start
2026-08-08 02:58:42.968 UTC [16458] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 02:58:42.968 UTC [16458] LOG: listening on IPv4 address "127.0.205.148", port 46973
2026-08-08 02:58:42.972 UTC [16458] LOG: listening on Unix socket "/tmp/.s.PGSQL.46973"
2026-08-08 02:58:42.994 UTC [16463] LOG: database system was shut down at 2026-08-08 02:58:41 UTC
2026-08-08 02:58:42.998 UTC [16458] LOG: database system is ready to accept connections
I20260808 02:58:44.217332 822 mini_postgres.cc:96] Postgres bound to 46973
2026-08-08 02:58:44.243 UTC [16476] FATAL: database "slave" does not exist
127.0.205.148:46973 - accepting connections
I20260808 02:58:44.244027 822 mini_ranger.cc:161] Starting Ranger...
I20260808 02:58:44.278322 822 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 02:58:44.834009 822 mini_ranger.cc:87] Created ranger Postgres database
I20260808 02:58:44.834182 822 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-admin
2026-08-08 02:58:45,164 [I] DB FLAVOR :POSTGRES
2026-08-08 02:58:45,165 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:58:45,165 [I] Checking connection..
2026-08-08 02:58:45,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 02:58:45,449 [I] Checking connection passed.
2026-08-08 02:58:45,449 [I] --------- Verifying version history table ---------
2026-08-08 02:58:45,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:58:45,724 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:58:45,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:58:45,992 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:58:45,993 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 02:58:45,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 02:58:46.264 UTC [16574] WARNING: there is no transaction in progress
2026-08-08 02:58:46,273 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 02:58:46,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:58:46,548 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 02:58:46,548 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:58:46,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 02:58:46,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 02:58:47,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6ntk.c.gcp-upstream.internal','N') ;"
2026-08-08 02:58:47,338 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:58:47,617 [I] Table x_portal_user does not exist in database ranger
2026-08-08 02:58:47,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:58:47,891 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 02:58:47,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:58:48,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:58:48,410 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 02:58:48,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 02:58:49.429 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.448 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.466 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.483 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.500 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.525 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.593 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.601 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.617 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.629 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.639 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.654 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.667 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.674 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.681 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:49.688 UTC [16752] WARNING: there is no transaction in progress
2026-08-08 02:58:50,414 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 02:58:50,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 02:58:50,683 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 02:58:50,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:58:50,961 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 02:58:50,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:58:51,239 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 02:58:51,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:58:51,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:58:51,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6ntk.c.gcp-upstream.internal';"
2026-08-08 02:58:52,063 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 02:58:52,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 02:58:52,307 [I] DB_PATCHES have already been applied
I20260808 02:58:52.313714 822 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvzvYzW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 02:58:52.313822 822 mini_ranger.cc:193] Using host: 127.0.205.148
I20260808 02:58:52.336184 822 mini_ranger.cc:227] Ranger admin URL: http://127.0.205.148:56127
Aug 08, 2026 2:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:58:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:58:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:58:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 02:58:53.755991 822 mini_ranger.cc:160] Time spent starting Ranger: real 9.512s user 0.010s sys 0.059s
Aug 08, 2026 2:59:14 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:59:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 2:59:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:59:14 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:59:15 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 02:59:15.733569 822 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.205.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.205.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/rangerkms_127.0.205.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/rangerkms_127.0.205.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.205.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.205.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.205.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/HTTP_127.0.205.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.205.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/HTTP_127.0.205.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 02:59:15 dist-test-slave-6ntk krb5kdc[16413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157955, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 02:59:15.955953 822 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 02:59:15.985462 822 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 02:59:16.946441 822 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 02:59:16.946635 822 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-kms
I20260808 02:59:16.946765 822 mini_ranger_kms.cc:227] Using postgres at 127.0.205.148:46973
2026-08-08 02:59:17,486 [I] DB FLAVOR :POSTGRES
2026-08-08 02:59:17,486 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:59:17,486 [I] Checking connection
2026-08-08 02:59:17,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 02:59:17,758 [I] connection success
2026-08-08 02:59:17,758 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 02:59:17,758 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 02:59:17,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 02:59:18,012 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 02:59:18,012 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:59:18,012 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 02:59:18,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:46973/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 02:59:18,282 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 02:59:18.793560 822 mini_ranger_kms.cc:310] Created kms service
I20260808 02:59:18.990684 822 mini_ranger_kms.cc:326] Created kudu user
I20260808 02:59:19.037335 822 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 02:59:19 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157955, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 02:59:19.495798 822 mini_ranger_kms.cc:384] Added ranger policy
I20260808 02:59:19.497244 822 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 02:59:19.497329 822 mini_ranger_kms.cc:240] Using host: 127.0.205.148
I20260808 02:59:19.523409 822 mini_ranger_kms.cc:276] Ranger KMS PID: 17023
I20260808 02:59:19.523583 822 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.205.148:57925
Aug 08, 2026 2:59:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:59:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:59:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:59:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:59:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:59:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 02:59:21 dist-test-slave-6ntk krb5kdc[16413](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786157961, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.205.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 02:59:22 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157961, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.205.148@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 02:59:22.431119 822 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.475s user 0.135s sys 0.443s
I20260808 02:59:22.431470 822 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 02:59:22.431751 822 mini_ranger_kms.cc:401] 127.0.205.148:57925/kms/v1/keys
Aug 08 02:59:22 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157961, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.205.148@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
Aug 08 02:59:25 dist-test-slave-6ntk krb5kdc[16413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 02:59:25.097733 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:44377
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:44377
--ranger_config_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:57925/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:59:25.471025 17106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:59:25.471498 17106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:59:25.471673 17106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:59:25.479532 17106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:59:25.479701 17106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:59:25.479823 17106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:59:25.479923 17106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:59:25.480019 17106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:59:25.490847 17106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.205.148:57925/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:44377
--ranger_config_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:44377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:59:25.492277 17106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:59:25.493649 17106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:59:25.506635 17118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:59:25.506722 17116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:59:25.507006 17106 server_base.cc:1034] running on GCE node
W20260808 02:59:25.506713 17115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:59:25.508081 17106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:59:25.509586 17106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:59:25.510833 17106 hybrid_clock.cc:648] HybridClock initialized: now 1786157965510767 us; error 59 us; skew 500 ppm
I20260808 02:59:25.511216 17106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:59:25 dist-test-slave-6ntk krb5kdc[16413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157965, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:59:25.514572 17106 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 02:59:25.516214 17106 webserver.cc:460] Webserver started at http://127.0.205.190:35563/ using document root <none> and password file <none>
I20260808 02:59:25.516932 17106 fs_manager.cc:359] Metadata directory not provided
I20260808 02:59:25.517081 17106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:59:25.517416 17106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:59:25 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157965, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 02:59:25.584386 17106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "a7ac928635c5481abf8e5c21d06ee73c"
format_stamp: "Formatted at 2026-08-08 02:59:25 on dist-test-slave-6ntk"
server_key: "3c27002b45e5ed1ccac46e4c6489baef"
server_key_iv: "d0764ee3875f110317ca74bb085f3c8b"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:25.585306 17106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "a7ac928635c5481abf8e5c21d06ee73c"
format_stamp: "Formatted at 2026-08-08 02:59:25 on dist-test-slave-6ntk"
server_key: "3c27002b45e5ed1ccac46e4c6489baef"
server_key_iv: "d0764ee3875f110317ca74bb085f3c8b"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:25.591275 17106 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260808 02:59:25.614105 17127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:59:25.616142 17106 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260808 02:59:25.616369 17106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "a7ac928635c5481abf8e5c21d06ee73c"
format_stamp: "Formatted at 2026-08-08 02:59:25 on dist-test-slave-6ntk"
server_key: "3c27002b45e5ed1ccac46e4c6489baef"
server_key_iv: "d0764ee3875f110317ca74bb085f3c8b"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:25.616677 17106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:59:25.645984 17106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:59:25.651813 17106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:59:25.652186 17106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:59:25.676714 17106 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:44377
I20260808 02:59:25.676837 17218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:44377 every 8 connection(s)
I20260808 02:59:25.678656 17106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 02:59:25.682895 17220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:59:25.685424 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 17106
I20260808 02:59:25.685639 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 02:59:25.686604 822 external_mini_cluster.cc:1428] Setting key 160d2a016fcfc736e0ee44664ea390c5
I20260808 02:59:25.699136 17220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c: Bootstrap starting.
I20260808 02:59:25.704161 17220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c: Neither blocks nor log segments found. Creating new log.
I20260808 02:59:25.705699 17220 log.cc:824] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:59:25.710011 17220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c: No bootstrap required, opened a new log
Aug 08 02:59:25 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:59:25.719472 17220 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ac928635c5481abf8e5c21d06ee73c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } }
I20260808 02:59:25.719902 17220 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:59:25.720167 17220 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ac928635c5481abf8e5c21d06ee73c, State: Initialized, Role: FOLLOWER
I20260808 02:59:25.720919 17220 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ac928635c5481abf8e5c21d06ee73c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } }
I20260808 02:59:25.721289 17220 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:59:25.721505 17220 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:59:25.721719 17220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:59:25.724956 17220 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ac928635c5481abf8e5c21d06ee73c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } }
I20260808 02:59:25.725623 17220 leader_election.cc:302] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7ac928635c5481abf8e5c21d06ee73c; no voters:
I20260808 02:59:25.726397 17220 leader_election.cc:288] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:59:25.726534 17227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:59:25.730567 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:25.688977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.0.205.190:44377)
0808 02:59:25.701502 (+ 12525us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:25.701518 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:25.702206 (+ 688us) server_negotiation.cc:407] Connection header received
0808 02:59:25.705149 (+ 2943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:25.705200 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:25.705769 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:25.706335 (+ 566us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:59:25.708047 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:25.710125 (+ 2078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:25.712188 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:25.713322 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:25.713547 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:25.718941 (+ 5394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:59:25.718954 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:59:25.718979 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:59:25.719036 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:59:25.723563 (+ 4527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:25.724524 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:25.724535 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:25.724546 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:25.724784 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:25.725293 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:25.725299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:25.725302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:25.726159 (+ 857us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:59:25.726497 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:25.728171 (+ 1674us) server_negotiation.cc:299] Negotiation successful
0808 02:59:25.728726 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12248,"thread_start_us":11985,"threads_started":1}
I20260808 02:59:25.736039 17220 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:59:25.737525 17227 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [term 1 LEADER]: Becoming Leader. State: Replica: a7ac928635c5481abf8e5c21d06ee73c, State: Running, Role: LEADER
I20260808 02:59:25.738471 17227 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ac928635c5481abf8e5c21d06ee73c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } }
I20260808 02:59:25.743733 17230 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7ac928635c5481abf8e5c21d06ee73c. Latest consensus state: current_term: 1 leader_uuid: "a7ac928635c5481abf8e5c21d06ee73c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ac928635c5481abf8e5c21d06ee73c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } } }
I20260808 02:59:25.744359 17230 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [sys.catalog]: This master's current role is: LEADER
I20260808 02:59:25.743695 17228 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7ac928635c5481abf8e5c21d06ee73c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ac928635c5481abf8e5c21d06ee73c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44377 } } }
I20260808 02:59:25.745035 17228 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c [sys.catalog]: This master's current role is: LEADER
I20260808 02:59:25.750557 17239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:59:25.758627 17239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:59:25.782811 17239 catalog_manager.cc:1347] Generated new cluster ID: 07202e22e04c4c358231c36e4f2112bb
I20260808 02:59:25.782986 17239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:59:25.797102 17239 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:59:25.799695 17239 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:59:25.801831 17220 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:59:25.834746 17239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c: Generated new TSK 0
I20260808 02:59:25.836171 17239 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 02:59:26.652426 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:57925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:59:26 dist-test-slave-6ntk krb5kdc[16413](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786157966, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 02:59:27.152290 17291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:59:27.152779 17291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:59:27.152967 17291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:59:27.166400 17291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:59:27.166580 17291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:59:27.166810 17291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 02:59:27.185010 17291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.205.148:57925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:59:27.187297 17291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:59:27.189342 17291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:59:27.208153 17305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:59:27.208778 17304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:59:27.222245 17291 server_base.cc:1034] running on GCE node
W20260808 02:59:27.223217 17308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:59:27.224143 17291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:59:27.225510 17291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:59:27.226836 17291 hybrid_clock.cc:648] HybridClock initialized: now 1786157967226706 us; error 117 us; skew 500 ppm
I20260808 02:59:27.227244 17291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:59:27 dist-test-slave-6ntk krb5kdc[16413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:59:27.236640 17291 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 02:59:27.238651 17291 webserver.cc:460] Webserver started at http://127.0.205.129:42075/ using document root <none> and password file <none>
I20260808 02:59:27.239612 17291 fs_manager.cc:359] Metadata directory not provided
I20260808 02:59:27.239768 17291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:59:27.240129 17291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:59:27 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 02:59:27.260535 17291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "b9f64066df1545d490c61c23e4e52cc2"
format_stamp: "Formatted at 2026-08-08 02:59:27 on dist-test-slave-6ntk"
server_key: "a2a75b7c1b4aed9883aa364e2aaf1519"
server_key_iv: "2be94088e15694b283bfd66dd27fdc26"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:27.261861 17291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "b9f64066df1545d490c61c23e4e52cc2"
format_stamp: "Formatted at 2026-08-08 02:59:27 on dist-test-slave-6ntk"
server_key: "a2a75b7c1b4aed9883aa364e2aaf1519"
server_key_iv: "2be94088e15694b283bfd66dd27fdc26"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:27.285987 17291 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.005s sys 0.015s
I20260808 02:59:27.310549 17322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:59:27.312603 17291 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260808 02:59:27.312835 17291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "b9f64066df1545d490c61c23e4e52cc2"
format_stamp: "Formatted at 2026-08-08 02:59:27 on dist-test-slave-6ntk"
server_key: "a2a75b7c1b4aed9883aa364e2aaf1519"
server_key_iv: "2be94088e15694b283bfd66dd27fdc26"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:27.313167 17291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:59:27.358197 17291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:59:27.364305 17291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:59:27.364782 17291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:59:27.366349 17291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:59:27.368119 17291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:59:27.368244 17291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 02:59:27.368388 17291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:59:27.368463 17291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:59:27.427767 17291 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:34499
I20260808 02:59:27.427790 17496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:34499 every 8 connection(s)
I20260808 02:59:27.429607 17291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 02:59:27.434430 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 17291
I20260808 02:59:27.434692 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 02:59:27.435798 822 external_mini_cluster.cc:1428] Setting key 888d71563160c7b2a9801c6400853f33
Aug 08 02:59:27 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:59:27.477941 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:27.440132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:42095 (local address 127.0.205.190:44377)
0808 02:59:27.454124 (+ 13992us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:27.454133 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:27.454163 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:59:27.454250 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:27.454259 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:27.454364 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:27.457544 (+ 3180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:59:27.457741 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:27.459013 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:27.461927 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:27.462597 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:27.462751 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:27.467940 (+ 5189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:59:27.467951 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:59:27.467959 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:59:27.468027 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:59:27.472269 (+ 4242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:27.473477 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:27.473486 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:27.473493 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:27.473673 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:27.474293 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:27.474301 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:27.474307 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:27.474738 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:59:27.474895 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:27.477302 (+ 2407us) server_negotiation.cc:299] Negotiation successful
0808 02:59:27.477555 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13812,"thread_start_us":14452,"threads_started":1}
I20260808 02:59:27.478093 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:27.439486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:44377 (local address 127.0.205.129:42095)
0808 02:59:27.440453 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0808 02:59:27.440503 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:59:27.447397 (+ 6894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:59:27.455507 (+ 8110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:59:27.455524 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:59:27.456350 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:59:27.457300 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:27.457331 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:27.459224 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:27.459232 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:27.461643 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:27.461662 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:27.463113 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:27.463123 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:27.463463 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:27.464417 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:59:27.464460 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:59:27.467603 (+ 3143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:59:27.472501 (+ 4898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:59:27.472513 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:59:27.472537 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:59:27.473272 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:59:27.473818 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:59:27.473826 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:59:27.473845 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:59:27.474150 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:59:27.474988 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:59:27.475000 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:59:27.475540 (+ 540us) client_negotiation.cc:769] Sending connection context
0808 02:59:27.475886 (+ 346us) client_negotiation.cc:240] Negotiation successful
0808 02:59:27.476503 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":704,"threads_started":1}
I20260808 02:59:27.480847 17497 heartbeater.cc:347] Connected to a master server at 127.0.205.190:44377
I20260808 02:59:27.481195 17497 heartbeater.cc:464] Registering TS with master...
I20260808 02:59:27.482154 17497 heartbeater.cc:511] Master 127.0.205.190:44377 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
I20260808 02:59:27.484956 17159 ts_manager.cc:194] Registered new tserver with Master: b9f64066df1545d490c61c23e4e52cc2 (127.0.205.129:34499)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
I20260808 02:59:27.487123 17159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:42095
Aug 08 02:59:27 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157966, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
W20260808 02:59:27.547070 822 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.205.130: real 0.108s user 0.000s sys 0.024s
I20260808 02:59:27.548065 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:57925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:59:27 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157966, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
W20260808 02:59:28.097970 17509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:59:28.098529 17509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:59:28.098762 17509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:59:28.110846 17509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:59:28.111137 17509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:59:28.111423 17509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 02:59:28.127628 17509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.205.148:57925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:59:28.129994 17509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:59:28.132146 17509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:59:28.168292 17527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:59:28.172286 17525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:59:28.183135 17532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:59:28.185101 17509 server_base.cc:1034] running on GCE node
I20260808 02:59:28.186355 17509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:59:28.196292 17509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:59:28.203116 17509 hybrid_clock.cc:648] HybridClock initialized: now 1786157968202674 us; error 437 us; skew 500 ppm
I20260808 02:59:28.203694 17509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:59:28 dist-test-slave-6ntk krb5kdc[16413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:59:28.208884 17509 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 02:59:28.218027 17509 webserver.cc:460] Webserver started at http://127.0.205.130:32981/ using document root <none> and password file <none>
I20260808 02:59:28.219255 17509 fs_manager.cc:359] Metadata directory not provided
I20260808 02:59:28.219496 17509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:59:28.219987 17509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:59:28 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 02:59:28.252111 17509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "c73617cf7ba146218f4f8a16934d86ca"
format_stamp: "Formatted at 2026-08-08 02:59:28 on dist-test-slave-6ntk"
server_key: "03c6093a1c6cb9018b2e1536155fe8e1"
server_key_iv: "a08c0b348fcaeb330a059fe2d92bb5ed"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:28.253217 17509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "c73617cf7ba146218f4f8a16934d86ca"
format_stamp: "Formatted at 2026-08-08 02:59:28 on dist-test-slave-6ntk"
server_key: "03c6093a1c6cb9018b2e1536155fe8e1"
server_key_iv: "a08c0b348fcaeb330a059fe2d92bb5ed"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:28.272625 17509 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.004s sys 0.002s
I20260808 02:59:28.300496 17542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:59:28.311566 17509 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.005s sys 0.004s
I20260808 02:59:28.311853 17509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "c73617cf7ba146218f4f8a16934d86ca"
format_stamp: "Formatted at 2026-08-08 02:59:28 on dist-test-slave-6ntk"
server_key: "03c6093a1c6cb9018b2e1536155fe8e1"
server_key_iv: "a08c0b348fcaeb330a059fe2d92bb5ed"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:28.312186 17509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:59:28.339107 17509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:59:28.344146 17509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:59:28.344522 17509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:59:28.345638 17509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:59:28.347420 17509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:59:28.347545 17509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:59:28.347707 17509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:59:28.347786 17509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:59:28.390542 17509 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:36777
I20260808 02:59:28.390561 17746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:36777 every 8 connection(s)
I20260808 02:59:28.391978 17509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 02:59:28.394079 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 17509
I20260808 02:59:28.394297 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 02:59:28.394882 822 external_mini_cluster.cc:1428] Setting key 29ec23103646932ba1043f1c3f75c2cb
Aug 08 02:59:28 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
I20260808 02:59:28.436476 17749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:28.394897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:57281 (local address 127.0.205.190:44377)
0808 02:59:28.410181 (+ 15284us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:28.410188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:28.410216 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:59:28.410723 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:28.410730 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:28.410834 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:28.410964 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:59:28.412995 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:28.414225 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:28.417400 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:28.418321 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:28.418484 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:28.424181 (+ 5697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:59:28.424188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:59:28.424193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:59:28.424241 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:59:28.428151 (+ 3910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:28.433326 (+ 5175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:28.433333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:28.433336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:28.433505 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:28.434107 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:28.434112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:28.434115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:28.434539 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:59:28.434689 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:28.435849 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0808 02:59:28.436125 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15137,"thread_start_us":15308,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
I20260808 02:59:28.437733 17750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:28.395697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:44377 (local address 127.0.205.130:57281)
0808 02:59:28.406741 (+ 11044us) negotiation.cc:107] Waiting for socket to connect
0808 02:59:28.406826 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0808 02:59:28.410040 (+ 3214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:59:28.411313 (+ 1273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:59:28.411330 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:59:28.412079 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:59:28.412781 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:28.412802 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:28.414414 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:28.414424 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:28.417130 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:28.417162 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:28.418653 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:28.418661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:28.419016 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:28.420216 (+ 1200us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:59:28.420252 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:59:28.423827 (+ 3575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:59:28.432395 (+ 8568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:59:28.432407 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:59:28.432426 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:59:28.433139 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:59:28.433648 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:59:28.433653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:59:28.433657 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:59:28.433955 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:59:28.434788 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:59:28.434796 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:59:28.435222 (+ 426us) client_negotiation.cc:769] Sending connection context
0808 02:59:28.435630 (+ 408us) client_negotiation.cc:240] Negotiation successful
0808 02:59:28.436301 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10617,"thread_start_us":10282,"threads_started":1}
I20260808 02:59:28.439740 17747 heartbeater.cc:347] Connected to a master server at 127.0.205.190:44377
I20260808 02:59:28.440092 17747 heartbeater.cc:464] Registering TS with master...
I20260808 02:59:28.440990 17747 heartbeater.cc:511] Master 127.0.205.190:44377 requested a full tablet report, sending...
I20260808 02:59:28.442891 17159 ts_manager.cc:194] Registered new tserver with Master: c73617cf7ba146218f4f8a16934d86ca (127.0.205.130:36777)
I20260808 02:59:28.444310 17159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:57281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 02:59:28.476881 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:57925/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:45697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:59:28.491365 17497 heartbeater.cc:503] Master 127.0.205.190:44377 was elected leader, sending a full tablet report...
W20260808 02:59:28.848980 17753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:59:28.849367 17753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:59:28.849531 17753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:59:28.857159 17753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:59:28.857328 17753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:59:28.857508 17753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 02:59:28.867890 17753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:45697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.205.148:57925/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 02:59:28.869330 17753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:59:28.870819 17753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:59:28.885141 17762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:59:28.885186 17764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:59:28.885210 17759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:59:28.886425 17753 server_base.cc:1034] running on GCE node
I20260808 02:59:28.887070 17753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:59:28.888088 17753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:59:28.889320 17753 hybrid_clock.cc:648] HybridClock initialized: now 1786157968889256 us; error 59 us; skew 500 ppm
I20260808 02:59:28.889664 17753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 02:59:28 dist-test-slave-6ntk krb5kdc[16413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:59:28.893008 17753 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 02:59:28.894661 17753 webserver.cc:460] Webserver started at http://127.0.205.131:37379/ using document root <none> and password file <none>
I20260808 02:59:28.895444 17753 fs_manager.cc:359] Metadata directory not provided
I20260808 02:59:28.895561 17753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:59:28.895848 17753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:59:28 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 02:59:28.914354 17753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "4bcc7681829043ba9ce67e5d37a8c05d"
format_stamp: "Formatted at 2026-08-08 02:59:28 on dist-test-slave-6ntk"
server_key: "355894aa3764125574350e12852671db"
server_key_iv: "843e22de83f3f00aedf6b84d27734ef7"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:28.915285 17753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "4bcc7681829043ba9ce67e5d37a8c05d"
format_stamp: "Formatted at 2026-08-08 02:59:28 on dist-test-slave-6ntk"
server_key: "355894aa3764125574350e12852671db"
server_key_iv: "843e22de83f3f00aedf6b84d27734ef7"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:28.921397 17753 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260808 02:59:28.936932 17775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:59:28.938655 17753 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 02:59:28.938859 17753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "4bcc7681829043ba9ce67e5d37a8c05d"
format_stamp: "Formatted at 2026-08-08 02:59:28 on dist-test-slave-6ntk"
server_key: "355894aa3764125574350e12852671db"
server_key_iv: "843e22de83f3f00aedf6b84d27734ef7"
server_key_version: "kuduclusterkey@0"
I20260808 02:59:28.939113 17753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:59:28.950793 17753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:59:28.954900 17753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:59:28.955222 17753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:59:28.956360 17753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:59:28.957806 17753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:59:28.957938 17753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 02:59:28.958107 17753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:59:28.958160 17753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:59:28.993618 17753 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:37203
I20260808 02:59:28.993657 17954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:37203 every 8 connection(s)
I20260808 02:59:28.994980 17753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 02:59:28.998236 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 17753
I20260808 02:59:28.998430 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 02:59:28.998891 822 external_mini_cluster.cc:1428] Setting key 1f72be801d4e387f5e1f2438af0c5bf1
Aug 08 02:59:29 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 02:59:29.016121 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:28.997857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:60673 (local address 127.0.205.190:44377)
0808 02:59:28.998518 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:28.998524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:28.999661 (+ 1137us) server_negotiation.cc:407] Connection header received
0808 02:59:29.001523 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.001529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.001641 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.001768 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:59:29.003402 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.004184 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.005728 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.006226 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.006331 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.009703 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:59:29.009708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:59:29.009711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:59:29.009747 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:59:29.012651 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:29.013631 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:29.013637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:29.013641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:29.013807 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:29.014435 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:29.014441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:29.014444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:29.014836 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:59:29.014956 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.015685 (+ 729us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.015860 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":309,"threads_started":1}
I20260808 02:59:29.017130 17961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:28.998534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:44377 (local address 127.0.205.131:60673)
0808 02:59:28.999460 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0808 02:59:28.999508 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 02:59:29.001255 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:59:29.002057 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:59:29.002066 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:59:29.002598 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:59:29.003242 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.003260 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.004321 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.004324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.005585 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.005605 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.006430 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.006435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.006650 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.007303 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:59:29.007326 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:59:29.009492 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:59:29.012814 (+ 3322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:59:29.012822 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:59:29.012835 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:59:29.013507 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:59:29.013987 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:59:29.013993 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:59:29.013996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:59:29.014290 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:59:29.014975 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:59:29.014981 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:59:29.015319 (+ 338us) client_negotiation.cc:769] Sending connection context
0808 02:59:29.015577 (+ 258us) client_negotiation.cc:240] Negotiation successful
0808 02:59:29.015973 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":427,"threads_started":1}
I20260808 02:59:29.018641 17956 heartbeater.cc:347] Connected to a master server at 127.0.205.190:44377
I20260808 02:59:29.018924 17956 heartbeater.cc:464] Registering TS with master...
I20260808 02:59:29.019680 17956 heartbeater.cc:511] Master 127.0.205.190:44377 requested a full tablet report, sending...
I20260808 02:59:29.021181 17159 ts_manager.cc:194] Registered new tserver with Master: 4bcc7681829043ba9ce67e5d37a8c05d (127.0.205.131:37203)
I20260808 02:59:29.022042 17159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:60673
I20260808 02:59:29.026902 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:59:29.044744 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.031509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36744 (local address 127.0.205.190:44377)
0808 02:59:29.031695 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:29.031703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:29.032171 (+ 468us) server_negotiation.cc:407] Connection header received
0808 02:59:29.032371 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.032378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.032471 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.032579 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:59:29.034133 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.034865 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.036210 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.036698 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.036843 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.039395 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:59:29.039407 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:59:29.039410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:59:29.039443 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:59:29.041923 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:29.042608 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:29.042615 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:29.042618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:29.042757 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:29.043187 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:29.043196 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:29.043198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:29.043555 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:59:29.043813 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.044183 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.044424 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:59:29.048815 17159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:59:29.056461 17159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:59:29.082623 17977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.069661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:36777 (local address 127.0.0.1:33928)
0808 02:59:29.071094 (+ 1433us) negotiation.cc:107] Waiting for socket to connect
0808 02:59:29.071131 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:59:29.071308 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:59:29.073524 (+ 2216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:59:29.073536 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:59:29.073580 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:59:29.075035 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.075043 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.077067 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.077073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.079474 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.079492 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.081505 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.081513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.081720 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.081775 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 02:59:29.081944 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 02:59:29.082151 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"thread_start_us":1027,"threads_started":1}
I20260808 02:59:29.083226 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.069676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.0.205.130:36777)
0808 02:59:29.071781 (+ 2105us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:29.071791 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:29.071820 (+ 29us) server_negotiation.cc:407] Connection header received
0808 02:59:29.071963 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.071972 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.072915 (+ 943us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.073164 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:59:29.075229 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.076888 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.079649 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.080966 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.081103 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.081323 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.082677 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.082892 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1934,"thread_start_us":1827,"threads_started":1}
I20260808 02:59:29.083621 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.072598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35058 (local address 127.0.205.129:34499)
0808 02:59:29.075126 (+ 2528us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:29.075135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:29.075162 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:59:29.075262 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.075271 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.075495 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.075750 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:59:29.076317 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.078454 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.080848 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.082254 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.082406 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.082639 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.083080 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.083259 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2359,"thread_start_us":1529,"threads_started":1}
I20260808 02:59:29.084424 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.069708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:37203 (local address 127.0.0.1:47446)
0808 02:59:29.074511 (+ 4803us) negotiation.cc:107] Waiting for socket to connect
0808 02:59:29.074528 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:59:29.074689 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:59:29.075352 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:59:29.075357 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:59:29.075385 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:59:29.075790 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.075802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.077770 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.077775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.080159 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.080174 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.082365 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.082373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.082722 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.082780 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 02:59:29.082876 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:59:29.083031 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4689,"mutex_wait_us":146,"thread_start_us":3115,"threads_started":1}
I20260808 02:59:29.084513 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.070352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47446 (local address 127.0.205.131:37203)
0808 02:59:29.072355 (+ 2003us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:29.072364 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:29.074652 (+ 2288us) server_negotiation.cc:407] Connection header received
0808 02:59:29.074887 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.074897 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.075136 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.075346 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:59:29.076009 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.077617 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.080338 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.081787 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.081982 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.082205 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.084050 (+ 1845us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.084191 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1762,"thread_start_us":1582,"threads_started":1}
I20260808 02:59:29.084472 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.072234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:34499 (local address 127.0.0.1:35058)
0808 02:59:29.074611 (+ 2377us) negotiation.cc:107] Waiting for socket to connect
0808 02:59:29.074631 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:59:29.074721 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:59:29.075756 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:59:29.075761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:59:29.075790 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:59:29.076152 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.076160 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.078635 (+ 2475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.078640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.080648 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.080658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.082528 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.082535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.082781 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.082807 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 02:59:29.082879 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:59:29.083554 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2243,"mutex_wait_us":549,"thread_start_us":2085,"threads_started":1}
I20260808 02:59:29.087507 17631 tablet_service.cc:1467] Processing CreateTablet for tablet 7ecd2860026d4ce5898d15c8b550e440 (DEFAULT_TABLE table=test-table [id=12ec610503fe491a93a7691256d1335b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:59:29.088835 17631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ecd2860026d4ce5898d15c8b550e440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:59:29.088526 17850 tablet_service.cc:1467] Processing CreateTablet for tablet 7ecd2860026d4ce5898d15c8b550e440 (DEFAULT_TABLE table=test-table [id=12ec610503fe491a93a7691256d1335b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:59:29.088526 17403 tablet_service.cc:1467] Processing CreateTablet for tablet 7ecd2860026d4ce5898d15c8b550e440 (DEFAULT_TABLE table=test-table [id=12ec610503fe491a93a7691256d1335b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:59:29.090025 17850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ecd2860026d4ce5898d15c8b550e440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:59:29.090024 17403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ecd2860026d4ce5898d15c8b550e440. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:59:29.101658 17983 tablet_bootstrap.cc:492] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: Bootstrap starting.
I20260808 02:59:29.105406 17985 tablet_bootstrap.cc:492] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: Bootstrap starting.
I20260808 02:59:29.105571 17983 tablet_bootstrap.cc:654] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: Neither blocks nor log segments found. Creating new log.
I20260808 02:59:29.106626 17986 tablet_bootstrap.cc:492] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: Bootstrap starting.
I20260808 02:59:29.108026 17983 log.cc:824] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: Log is configured to *not* fsync() on all Append() calls
I20260808 02:59:29.109750 17985 tablet_bootstrap.cc:654] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: Neither blocks nor log segments found. Creating new log.
I20260808 02:59:29.110580 17986 tablet_bootstrap.cc:654] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: Neither blocks nor log segments found. Creating new log.
I20260808 02:59:29.111024 17985 log.cc:824] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: Log is configured to *not* fsync() on all Append() calls
I20260808 02:59:29.111874 17986 log.cc:824] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:59:29.112349 17983 tablet_bootstrap.cc:492] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: No bootstrap required, opened a new log
I20260808 02:59:29.112701 17983 ts_tablet_manager.cc:1397] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260808 02:59:29.114248 17985 tablet_bootstrap.cc:492] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: No bootstrap required, opened a new log
I20260808 02:59:29.114521 17985 ts_tablet_manager.cc:1397] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 02:59:29.115223 17986 tablet_bootstrap.cc:492] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: No bootstrap required, opened a new log
I20260808 02:59:29.115499 17986 ts_tablet_manager.cc:1397] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 02:59:29.122897 17985 raft_consensus.cc:348] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 } } peers { permanent_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 37203 } } peers { permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 } }
I20260808 02:59:29.123303 17985 raft_consensus.cc:374] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:59:29.123446 17985 raft_consensus.cc:729] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9f64066df1545d490c61c23e4e52cc2, State: Initialized, Role: FOLLOWER
I20260808 02:59:29.123793 17983 raft_consensus.cc:348] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 } } peers { permanent_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 37203 } } peers { permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 } }
I20260808 02:59:29.124222 17983 raft_consensus.cc:374] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:59:29.124325 17983 raft_consensus.cc:729] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73617cf7ba146218f4f8a16934d86ca, State: Initialized, Role: FOLLOWER
I20260808 02:59:29.124109 17985 consensus_queue.cc:260] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 } } peers { permanent_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 37203 } } peers { permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 } }
I20260808 02:59:29.124913 17983 consensus_queue.cc:260] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 } } peers { permanent_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 37203 } } peers { permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 } }
I20260808 02:59:29.126210 17985 ts_tablet_manager.cc:1428] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 02:59:29.126602 17986 raft_consensus.cc:348] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 } } peers { permanent_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 37203 } } peers { permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 } }
I20260808 02:59:29.127128 17986 raft_consensus.cc:374] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:59:29.127261 17986 raft_consensus.cc:729] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcc7681829043ba9ce67e5d37a8c05d, State: Initialized, Role: FOLLOWER
I20260808 02:59:29.127450 17747 heartbeater.cc:503] Master 127.0.205.190:44377 was elected leader, sending a full tablet report...
I20260808 02:59:29.128175 17983 ts_tablet_manager.cc:1428] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: Time spent starting tablet: real 0.015s user 0.007s sys 0.007s
I20260808 02:59:29.128018 17986 consensus_queue.cc:260] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 } } peers { permanent_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 37203 } } peers { permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 } }
I20260808 02:59:29.129159 17956 heartbeater.cc:503] Master 127.0.205.190:44377 was elected leader, sending a full tablet report...
I20260808 02:59:29.131129 17986 ts_tablet_manager.cc:1428] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260808 02:59:29.144225 17748 tablet.cc:2369] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:59:29.146324 17993 raft_consensus.cc:482] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:59:29.146605 17993 raft_consensus.cc:504] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 } } peers { permanent_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 37203 } } peers { permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 } }
I20260808 02:59:29.148268 17993 leader_election.cc:288] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c73617cf7ba146218f4f8a16934d86ca (127.0.205.130:36777), b9f64066df1545d490c61c23e4e52cc2 (127.0.205.129:34499)
I20260808 02:59:29.155594 17961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.148694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:36777 (local address 127.0.205.131:59699)
0808 02:59:29.148905 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 02:59:29.148923 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:59:29.149025 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:59:29.149427 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:59:29.149431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:59:29.149461 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:59:29.149694 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.149711 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.151695 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.151699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.153144 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.153157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.154750 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.154756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.154987 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.155018 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:59:29.155142 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 02:59:29.155317 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 02:59:29.155579 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.148769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:59699 (local address 127.0.205.130:36777)
0808 02:59:29.149005 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:29.149013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:29.149037 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:59:29.149185 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.149190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.149276 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.149386 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:59:29.149925 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.151466 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.153309 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.154592 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.154755 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.154827 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.155165 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.155289 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:59:29.156898 17665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecd2860026d4ce5898d15c8b550e440" candidate_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73617cf7ba146218f4f8a16934d86ca" is_pre_election: true
I20260808 02:59:29.157426 17665 raft_consensus.cc:2454] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcc7681829043ba9ce67e5d37a8c05d in term 0.
I20260808 02:59:29.158104 17777 leader_election.cc:302] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcc7681829043ba9ce67e5d37a8c05d, c73617cf7ba146218f4f8a16934d86ca; no voters:
I20260808 02:59:29.158538 17993 raft_consensus.cc:2781] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:59:29.158691 17993 raft_consensus.cc:482] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:59:29.158807 17993 raft_consensus.cc:3037] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:59:29.160509 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.149421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:43905 (local address 127.0.205.129:34499)
0808 02:59:29.149642 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:29.149649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:29.154461 (+ 4812us) server_negotiation.cc:407] Connection header received
0808 02:59:29.154649 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.154653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.154715 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.154809 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:59:29.155481 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.156505 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.158599 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.159541 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.159727 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.159805 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.160122 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.160255 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 02:59:29.160614 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.149163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:34499 (local address 127.0.205.131:43905)
0808 02:59:29.154339 (+ 5176us) negotiation.cc:107] Waiting for socket to connect
0808 02:59:29.154354 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 02:59:29.154499 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:59:29.154824 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:59:29.154836 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:59:29.154862 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:59:29.155276 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.155288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.156691 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.156696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.158444 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:59:29.158458 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.159772 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.159778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:59:29.159952 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.159979 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 02:59:29.160094 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 02:59:29.160234 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5079,"thread_start_us":4917,"threads_started":1}
I20260808 02:59:29.161459 17993 raft_consensus.cc:504] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 } } peers { permanent_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 37203 } } peers { permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 } }
I20260808 02:59:29.161777 17433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecd2860026d4ce5898d15c8b550e440" candidate_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9f64066df1545d490c61c23e4e52cc2" is_pre_election: true
I20260808 02:59:29.162326 17993 leader_election.cc:288] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [CANDIDATE]: Term 1 election: Requested vote from peers c73617cf7ba146218f4f8a16934d86ca (127.0.205.130:36777), b9f64066df1545d490c61c23e4e52cc2 (127.0.205.129:34499)
I20260808 02:59:29.162498 17433 raft_consensus.cc:2454] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcc7681829043ba9ce67e5d37a8c05d in term 0.
I20260808 02:59:29.162741 17665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecd2860026d4ce5898d15c8b550e440" candidate_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c73617cf7ba146218f4f8a16934d86ca"
I20260808 02:59:29.162967 17665 raft_consensus.cc:3037] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:59:29.163003 17432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ecd2860026d4ce5898d15c8b550e440" candidate_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9f64066df1545d490c61c23e4e52cc2"
I20260808 02:59:29.163264 17432 raft_consensus.cc:3037] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:59:29.166422 17665 raft_consensus.cc:2454] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcc7681829043ba9ce67e5d37a8c05d in term 1.
I20260808 02:59:29.166687 17432 raft_consensus.cc:2454] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcc7681829043ba9ce67e5d37a8c05d in term 1.
I20260808 02:59:29.166886 17777 leader_election.cc:302] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcc7681829043ba9ce67e5d37a8c05d, c73617cf7ba146218f4f8a16934d86ca; no voters:
I20260808 02:59:29.167227 17993 raft_consensus.cc:2781] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:59:29.167682 17993 raft_consensus.cc:686] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 1 LEADER]: Becoming Leader. State: Replica: 4bcc7681829043ba9ce67e5d37a8c05d, State: Running, Role: LEADER
I20260808 02:59:29.168514 17993 consensus_queue.cc:237] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 } } peers { permanent_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 37203 } } peers { permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 } }
I20260808 02:59:29.173321 17155 catalog_manager.cc:5495] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcc7681829043ba9ce67e5d37a8c05d (127.0.205.131). New cstate: current_term: 1 leader_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bcc7681829043ba9ce67e5d37a8c05d" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 37203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 } health_report { overall_health: UNKNOWN } } }
W20260808 02:59:29.206354 17502 tablet.cc:2369] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:59:29.214406 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.208872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47452 (local address 127.0.205.131:37203)
0808 02:59:29.209085 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:29.209089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:29.209108 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:59:29.209199 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.209203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.209282 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.209421 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:59:29.209810 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.210624 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.212064 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.212604 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.212703 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.213227 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:59:29.213779 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:59:29.213960 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.214032 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.214102 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:59:29.223248 17665 raft_consensus.cc:1261] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca [term 1 FOLLOWER]: Refusing update from remote peer 4bcc7681829043ba9ce67e5d37a8c05d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:59:29.223296 17432 raft_consensus.cc:1261] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2 [term 1 FOLLOWER]: Refusing update from remote peer 4bcc7681829043ba9ce67e5d37a8c05d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:59:29.224274 17999 consensus_queue.cc:1035] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c73617cf7ba146218f4f8a16934d86ca" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:59:29.224713 17993 consensus_queue.cc:1035] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9f64066df1545d490c61c23e4e52cc2" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:59:29.237089 18004 mvcc.cc:205] Tried to move back new op lower bound from 7316103041927319552 to 7316103041718296576. Current Snapshot: MvccSnapshot[applied={T|T < 7316103041927319552}]
I20260808 02:59:29.241688 18006 mvcc.cc:205] Tried to move back new op lower bound from 7316103041927319552 to 7316103041718296576. Current Snapshot: MvccSnapshot[applied={T|T < 7316103041927319552}]
I20260808 02:59:29.258919 17159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:59:29.259306 17159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:59:29.263509 17159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7ac928635c5481abf8e5c21d06ee73c: Sending DeleteTablet for 3 replicas of tablet 7ecd2860026d4ce5898d15c8b550e440
I20260808 02:59:29.264926 17403 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ecd2860026d4ce5898d15c8b550e440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:59:29 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:35058
I20260808 02:59:29.265076 17631 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ecd2860026d4ce5898d15c8b550e440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:59:29 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:33928
I20260808 02:59:29.265362 17850 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ecd2860026d4ce5898d15c8b550e440 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:59:29 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:47446
I20260808 02:59:29.267803 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 17291
I20260808 02:59:29.277478 18021 tablet_replica.cc:330] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: stopping tablet replica
I20260808 02:59:29.278096 18021 raft_consensus.cc:2229] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:59:29.279417 18024 tablet_replica.cc:330] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: stopping tablet replica
I20260808 02:59:29.279959 18024 raft_consensus.cc:2229] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:59:29.280597 18022 tablet_replica.cc:330] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: stopping tablet replica
I20260808 02:59:29.280920 18024 raft_consensus.cc:2258] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:59:29.281369 18022 raft_consensus.cc:2229] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:59:29.281567 18021 raft_consensus.cc:2258] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:59:29.282438 18022 raft_consensus.cc:2258] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:59:29.284658 18024 ts_tablet_manager.cc:1905] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:59:29.284943 18022 ts_tablet_manager.cc:1905] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:59:29.287381 18021 ts_tablet_manager.cc:1905] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:59:29.293090 18022 ts_tablet_manager.cc:1918] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:59:29.293356 18022 log.cc:1196] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/7ecd2860026d4ce5898d15c8b550e440
I20260808 02:59:29.293555 18024 ts_tablet_manager.cc:1918] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:59:29.294028 18022 ts_tablet_manager.cc:1939] T 7ecd2860026d4ce5898d15c8b550e440 P c73617cf7ba146218f4f8a16934d86ca: Deleting consensus metadata
I20260808 02:59:29.293859 18024 log.cc:1196] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/7ecd2860026d4ce5898d15c8b550e440
I20260808 02:59:29.294739 18021 ts_tablet_manager.cc:1918] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:59:29.294927 18021 log.cc:1196] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/7ecd2860026d4ce5898d15c8b550e440
I20260808 02:59:29.295259 18024 ts_tablet_manager.cc:1939] T 7ecd2860026d4ce5898d15c8b550e440 P 4bcc7681829043ba9ce67e5d37a8c05d: Deleting consensus metadata
I20260808 02:59:29.295477 18021 ts_tablet_manager.cc:1939] T 7ecd2860026d4ce5898d15c8b550e440 P b9f64066df1545d490c61c23e4e52cc2: Deleting consensus metadata
I20260808 02:59:29.296511 17129 catalog_manager.cc:4841] TS c73617cf7ba146218f4f8a16934d86ca (127.0.205.130:36777): tablet 7ecd2860026d4ce5898d15c8b550e440 (table test-table [id=12ec610503fe491a93a7691256d1335b]) successfully deleted
I20260808 02:59:29.297312 17131 catalog_manager.cc:4841] TS 4bcc7681829043ba9ce67e5d37a8c05d (127.0.205.131:37203): tablet 7ecd2860026d4ce5898d15c8b550e440 (table test-table [id=12ec610503fe491a93a7691256d1335b]) successfully deleted
I20260808 02:59:29.298245 17129 catalog_manager.cc:4841] TS b9f64066df1545d490c61c23e4e52cc2 (127.0.205.129:34499): tablet 7ecd2860026d4ce5898d15c8b550e440 (table test-table [id=12ec610503fe491a93a7691256d1335b]) successfully deleted
Aug 08 02:59:29 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 02:59:29.321959 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.269828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.0.205.129:34499)
0808 02:59:29.270100 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:29.270109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:29.283172 (+ 13063us) server_negotiation.cc:407] Connection header received
0808 02:59:29.295973 (+ 12801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.295980 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.296077 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.296156 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:59:29.298870 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.299786 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.301927 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.302590 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.302696 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.313499 (+ 10803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:59:29.313520 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:59:29.313543 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:59:29.313610 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:59:29.318254 (+ 4644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:29.319296 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:29.319306 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:29.319310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:29.319510 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:29.319991 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:29.319998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:29.320002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:29.320551 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:59:29.320883 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.321182 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.321461 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 02:59:29.322782 17492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:59:29.425971 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 17291
I20260808 02:59:29.454257 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 17509
Aug 08 02:59:29 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 02:59:29.471481 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.455206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33934 (local address 127.0.205.130:36777)
0808 02:59:29.455418 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:29.455424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:29.455447 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:59:29.455695 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.455700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.455775 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.455898 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:59:29.457355 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.458285 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.459753 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.460202 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.460353 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.465088 (+ 4735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:59:29.465103 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:59:29.465119 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:59:29.465156 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:59:29.468232 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:29.469035 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:29.469043 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:29.469045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:29.469181 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:29.469597 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:29.469602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:29.469605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:29.470200 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:59:29.470526 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.470808 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.471045 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 02:59:29.472162 17740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:59:29.565689 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 17509
I20260808 02:59:29.597425 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 17753
Aug 08 02:59:29 dist-test-slave-6ntk krb5kdc[16413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 02:59:29.616564 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:59:29.598512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47460 (local address 127.0.205.131:37203)
0808 02:59:29.598712 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 02:59:29.598721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:59:29.598749 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:59:29.598998 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:59:29.599004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:59:29.599085 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:59:29.599210 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:59:29.600673 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.601644 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.603205 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:59:29.603757 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:59:29.603922 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:59:29.609079 (+ 5157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:59:29.609100 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:59:29.609124 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:59:29.609179 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:59:29.613304 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:29.614222 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:29.614231 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:29.614233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:29.614433 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:59:29.614859 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:59:29.614864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:59:29.614866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:59:29.615255 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:59:29.615618 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0808 02:59:29.615932 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 02:59:29.616179 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 02:59:29.617323 17948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:59:29.756408 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 17753
I20260808 02:59:29.783159 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 17106
I20260808 02:59:29.784397 17206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 02:59:29.898897 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 17106
I20260808 02:59:29.933264 822 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 02:59:30.279520 822 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 02:59:30.279883 822 mini_ranger.cc:66] Stopping Ranger...
I20260808 02:59:30.627477 822 mini_ranger.cc:68] Stopped Ranger
2026-08-08 02:59:30.627 UTC [16458] LOG: received smart shutdown request
2026-08-08 02:59:30.631 UTC [16458] LOG: background worker "logical replication launcher" (PID 16471) exited with exit code 1
2026-08-08 02:59:30.632 UTC [16465] LOG: shutting down
2026-08-08 02:59:30.808 UTC [16458] LOG: database system is shut down
2026-08-08T02:59:30Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52784 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:59:31 dist-test-slave-6ntk krb5kdc[18049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:59:31 dist-test-slave-6ntk krb5kdc[18049](info): set up 2 sockets
Aug 08 02:59:31 dist-test-slave-6ntk krb5kdc[18049](info): commencing operation
krb5kdc: starting...
W20260808 02:59:32.279454 822 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 02:59:32 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786157972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:59:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:59:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.205.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.205.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/rangeradmin_127.0.205.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/rangeradmin_127.0.205.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.205.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.205.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/rangerlookup_127.0.205.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/rangerlookup_127.0.205.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.205.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.205.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/HTTP_127.0.205.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/HTTP_127.0.205.148@KRBTEST.COM.keytab.
I20260808 02:59:32.686841 822 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvzvYzW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/postgres -l logfile start
2026-08-08 02:59:35.352 UTC [18105] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 02:59:35.353 UTC [18105] LOG: listening on IPv4 address "127.0.205.148", port 53461
2026-08-08 02:59:35.356 UTC [18105] LOG: listening on Unix socket "/tmp/.s.PGSQL.53461"
2026-08-08 02:59:35.381 UTC [18109] LOG: database system was shut down at 2026-08-08 02:59:33 UTC
2026-08-08 02:59:35.385 UTC [18105] LOG: database system is ready to accept connections
I20260808 02:59:36.569758 822 mini_postgres.cc:96] Postgres bound to 53461
2026-08-08 02:59:36.594 UTC [18125] FATAL: database "slave" does not exist
127.0.205.148:53461 - accepting connections
I20260808 02:59:36.594825 822 mini_ranger.cc:161] Starting Ranger...
I20260808 02:59:36.628085 822 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 02:59:37.197263 822 mini_ranger.cc:87] Created ranger Postgres database
I20260808 02:59:37.197403 822 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-admin
2026-08-08 02:59:37,515 [I] DB FLAVOR :POSTGRES
2026-08-08 02:59:37,515 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:59:37,515 [I] Checking connection..
2026-08-08 02:59:37,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 02:59:37,787 [I] Checking connection passed.
2026-08-08 02:59:37,788 [I] --------- Verifying version history table ---------
2026-08-08 02:59:37,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:59:38,062 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:59:38,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:59:38,349 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:59:38,350 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 02:59:38,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 02:59:38.602 UTC [18224] WARNING: there is no transaction in progress
2026-08-08 02:59:38,611 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 02:59:38,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:59:38,921 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 02:59:38,921 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:59:38,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 02:59:39,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 02:59:39,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6ntk.c.gcp-upstream.internal','N') ;"
2026-08-08 02:59:39,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:59:39,962 [I] Table x_portal_user does not exist in database ranger
2026-08-08 02:59:39,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:59:40,220 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 02:59:40,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:59:40,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:59:40,750 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 02:59:40,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 02:59:41.771 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.789 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.807 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.824 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.840 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.861 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.929 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.936 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.954 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.966 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.977 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:41.990 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:42.000 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:42.007 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:42.014 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:42.022 UTC [18406] WARNING: there is no transaction in progress
2026-08-08 02:59:42,736 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 02:59:42,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 02:59:43,003 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 02:59:43,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:59:43,271 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 02:59:43,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:59:43,529 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 02:59:43,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:59:43,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:59:44,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6ntk.c.gcp-upstream.internal';"
2026-08-08 02:59:44,316 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 02:59:44,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 02:59:44,582 [I] DB_PATCHES have already been applied
I20260808 02:59:44.588218 822 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvzvYzW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 02:59:44.588341 822 mini_ranger.cc:193] Using host: 127.0.205.148
I20260808 02:59:44.609009 822 mini_ranger.cc:227] Ranger admin URL: http://127.0.205.148:54267
Aug 08, 2026 2:59:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:59:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:59:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:59:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:59:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:59:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:59:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 02:59:46.022979 822 mini_ranger.cc:160] Time spent starting Ranger: real 9.428s user 0.004s sys 0.057s
Aug 08, 2026 3:00:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 3:00:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 3:00:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 3:00:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 3:00:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 03:00:08.798810 822 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.205.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.205.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/rangerkms_127.0.205.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.205.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/rangerkms_127.0.205.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.205.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.205.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.205.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/HTTP_127.0.205.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.205.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/HTTP_127.0.205.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 03:00:09 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158009, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 03:00:09.019534 822 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 03:00:09.047853 822 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 03:00:10.042126 822 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 03:00:10.042264 822 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-kms
I20260808 03:00:10.042331 822 mini_ranger_kms.cc:227] Using postgres at 127.0.205.148:53461
2026-08-08 03:00:10,424 [I] DB FLAVOR :POSTGRES
2026-08-08 03:00:10,425 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 03:00:10,425 [I] Checking connection
2026-08-08 03:00:10,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 03:00:10,701 [I] connection success
2026-08-08 03:00:10,701 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 03:00:10,701 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 03:00:10,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 03:00:10,952 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 03:00:10,953 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 03:00:10,953 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 03:00:10,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.205.148:53461/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 03:00:11,223 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 03:00:11.786185 822 mini_ranger_kms.cc:310] Created kms service
I20260808 03:00:11.959836 822 mini_ranger_kms.cc:326] Created kudu user
I20260808 03:00:12.006901 822 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 03:00:12 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158009, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 03:00:12.420253 822 mini_ranger_kms.cc:384] Added ranger policy
I20260808 03:00:12.420701 822 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvzvYzW/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvzvYzW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 03:00:12.420769 822 mini_ranger_kms.cc:240] Using host: 127.0.205.148
I20260808 03:00:12.449158 822 mini_ranger_kms.cc:276] Ranger KMS PID: 18680
I20260808 03:00:12.449414 822 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.205.148:42885
Aug 08, 2026 3:00:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:00:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:00:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 3:00:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 3:00:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 3:00:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 03:00:14 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786158014, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.205.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 03:00:15 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158014, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.205.148@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 03:00:15.448874 822 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.429s user 0.071s sys 0.348s
I20260808 03:00:15.449124 822 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 03:00:15.449309 822 mini_ranger_kms.cc:401] 127.0.205.148:42885/kms/v1/keys
Aug 08 03:00:15 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158014, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.205.148@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
Aug 08 03:00:17 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:00:17.973428 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:41417
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:41417
--ranger_config_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:42885/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:18.336766 18757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:18.337174 18757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:18.337342 18757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:18.345029 18757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:00:18.345193 18757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:18.345310 18757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:18.345414 18757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:00:18.345504 18757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:00:18.356484 18757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.205.148:42885/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:41417
--ranger_config_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:41417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:18.357940 18757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:18.359364 18757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:18.372164 18766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:18.372159 18767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:18.372256 18771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:18.373322 18757 server_base.cc:1034] running on GCE node
I20260808 03:00:18.374284 18757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:18.375802 18757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:18.377059 18757 hybrid_clock.cc:648] HybridClock initialized: now 1786158018377014 us; error 56 us; skew 500 ppm
I20260808 03:00:18.377444 18757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:18 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:18.387203 18757 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:00:18.388870 18757 webserver.cc:460] Webserver started at http://127.0.205.190:36699/ using document root <none> and password file <none>
I20260808 03:00:18.389626 18757 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:18.389778 18757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:18.390167 18757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:00:18 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 03:00:18.450493 18757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "1affc4a6109a4af5abfb9bdac6e57da7"
format_stamp: "Formatted at 2026-08-08 03:00:18 on dist-test-slave-6ntk"
server_key: "0550a1e74efaac233b867dcdad5b01b9"
server_key_iv: "64f9e0e003dd45b0b215bfef0a6adbbf"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:18.451608 18757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "1affc4a6109a4af5abfb9bdac6e57da7"
format_stamp: "Formatted at 2026-08-08 03:00:18 on dist-test-slave-6ntk"
server_key: "0550a1e74efaac233b867dcdad5b01b9"
server_key_iv: "64f9e0e003dd45b0b215bfef0a6adbbf"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:18.458174 18757 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260808 03:00:18.476683 18778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:18.478526 18757 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 03:00:18.478780 18757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "1affc4a6109a4af5abfb9bdac6e57da7"
format_stamp: "Formatted at 2026-08-08 03:00:18 on dist-test-slave-6ntk"
server_key: "0550a1e74efaac233b867dcdad5b01b9"
server_key_iv: "64f9e0e003dd45b0b215bfef0a6adbbf"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:18.479055 18757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:18.513962 18757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:18.518877 18757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:18.519320 18757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:18.544004 18757 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:41417
I20260808 03:00:18.544036 18854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:41417 every 8 connection(s)
I20260808 03:00:18.545660 18757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:00:18.549862 18855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:18.556170 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 18757
I20260808 03:00:18.556438 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:00:18.557370 822 external_mini_cluster.cc:1428] Setting key 2f7a8bcd64d0860911ac57e787712b93
I20260808 03:00:18.562426 18855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: Bootstrap starting.
I20260808 03:00:18.566639 18855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:18.567891 18855 log.cc:824] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:18.571270 18855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: No bootstrap required, opened a new log
Aug 08 03:00:18 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:18.579942 18855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } }
I20260808 03:00:18.580386 18855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:18.580550 18855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1affc4a6109a4af5abfb9bdac6e57da7, State: Initialized, Role: FOLLOWER
I20260808 03:00:18.581281 18855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } }
I20260808 03:00:18.581584 18855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:00:18.581740 18855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:00:18.581957 18855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:18.583237 18860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:18.559902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.0.205.190:41417)
0808 03:00:18.560807 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:18.560820 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:18.560874 (+ 54us) server_negotiation.cc:407] Connection header received
0808 03:00:18.562821 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:18.562856 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:18.563302 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:18.563656 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:18.565567 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:18.567352 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:18.569220 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:18.570161 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:18.570296 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:18.575100 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:18.575109 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:18.575126 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:18.575169 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:18.578477 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:18.579234 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:18.579242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:18.579250 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:18.579405 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:18.579806 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:18.579812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:18.579815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:18.580438 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:18.580755 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:18.581278 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 03:00:18.581722 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":430,"threads_started":1}
I20260808 03:00:18.584959 18855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } }
I20260808 03:00:18.585506 18855 leader_election.cc:302] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1affc4a6109a4af5abfb9bdac6e57da7; no voters:
I20260808 03:00:18.586251 18855 leader_election.cc:288] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:00:18.586371 18863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:00:18.587244 18863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 1 LEADER]: Becoming Leader. State: Replica: 1affc4a6109a4af5abfb9bdac6e57da7, State: Running, Role: LEADER
I20260808 03:00:18.587862 18863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } }
I20260808 03:00:18.588510 18855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:00:18.591918 18867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } } }
I20260808 03:00:18.592448 18867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:18.593294 18868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1affc4a6109a4af5abfb9bdac6e57da7. Latest consensus state: current_term: 1 leader_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } } }
I20260808 03:00:18.593550 18868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:18.594192 18878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:00:18.599773 18878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:00:18.603296 18855 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 03:00:18.620378 18878 catalog_manager.cc:1347] Generated new cluster ID: 19e286a6f89a47209f08812602962831
I20260808 03:00:18.620548 18878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:00:18.647066 18878 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:00:18.649484 18878 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:00:18.673442 18878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: Generated new TSK 0
I20260808 03:00:18.674809 18878 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 03:00:19.426708 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:42885/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:41417
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:00:19 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786158019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 03:00:19.889837 18925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:19.890407 18925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:19.890606 18925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:19.902675 18925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:19.902906 18925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:19.903134 18925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:00:19.919667 18925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.205.148:42885/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:41417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:19.921679 18925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:19.923563 18925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:19.944196 18942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:19.944914 18941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:19.957036 18944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:19.964233 18925 server_base.cc:1034] running on GCE node
I20260808 03:00:19.965624 18925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:19.967680 18925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:19.969007 18925 hybrid_clock.cc:648] HybridClock initialized: now 1786158019968984 us; error 161 us; skew 500 ppm
I20260808 03:00:19.969465 18925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:19 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:19.974082 18925 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:00:19.982203 18925 webserver.cc:460] Webserver started at http://127.0.205.129:38085/ using document root <none> and password file <none>
I20260808 03:00:19.983214 18925 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:19.983405 18925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:19.983822 18925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:00:19 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 03:00:20.015938 18925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "a9ea99186d2d4b9aac3a13e2809a2e01"
format_stamp: "Formatted at 2026-08-08 03:00:19 on dist-test-slave-6ntk"
server_key: "53a942e276bc943613371648397e6d8c"
server_key_iv: "b4bb0ae071190fcea470791729882e46"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:20.017307 18925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "a9ea99186d2d4b9aac3a13e2809a2e01"
format_stamp: "Formatted at 2026-08-08 03:00:19 on dist-test-slave-6ntk"
server_key: "53a942e276bc943613371648397e6d8c"
server_key_iv: "b4bb0ae071190fcea470791729882e46"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:20.026623 18925 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260808 03:00:20.066928 18956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:20.071962 18925 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.002s sys 0.004s
I20260808 03:00:20.072281 18925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "a9ea99186d2d4b9aac3a13e2809a2e01"
format_stamp: "Formatted at 2026-08-08 03:00:19 on dist-test-slave-6ntk"
server_key: "53a942e276bc943613371648397e6d8c"
server_key_iv: "b4bb0ae071190fcea470791729882e46"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:20.072590 18925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:20.127385 18925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:20.132833 18925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:20.133275 18925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:20.134804 18925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:20.137140 18925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:20.137341 18925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 03:00:20.137569 18925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:20.137713 18925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 03:00:20 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
Aug 08 03:00:20 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 03:00:20.809405 18925 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:43859
I20260808 03:00:20.809440 19125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:43859 every 8 connection(s)
I20260808 03:00:20.811295 18925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:00:20.814018 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 18925
I20260808 03:00:20.814262 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 03:00:20.815176 822 external_mini_cluster.cc:1428] Setting key 798368c85c96be1c391d3c62135447a6
Aug 08 03:00:20 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
I20260808 03:00:20.882093 19128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:20.816154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41417 (local address 127.0.205.129:50595)
0808 03:00:20.819304 (+ 3150us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:20.819362 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:20.825171 (+ 5809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:20.849043 (+ 23872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:20.849060 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:20.849797 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:20.850800 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:20.850832 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:20.852604 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:20.852610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:20.854862 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:20.854879 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:20.866247 (+ 11368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:20.866262 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:20.866699 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:20.867739 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:20.867786 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:20.871051 (+ 3265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:20.876630 (+ 5579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:20.876642 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:20.876667 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:20.877409 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:20.877953 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:20.877958 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:20.877962 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:20.878285 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:20.879063 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:20.879073 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:20.879585 (+ 512us) client_negotiation.cc:769] Sending connection context
0808 03:00:20.879913 (+ 328us) client_negotiation.cc:240] Negotiation successful
0808 03:00:20.880581 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1883,"thread_start_us":13624,"threads_started":1}
I20260808 03:00:20.883364 19129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:20.817467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:50595 (local address 127.0.205.190:41417)
0808 03:00:20.848112 (+ 30645us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:20.848120 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:20.848149 (+ 29us) server_negotiation.cc:407] Connection header received
0808 03:00:20.848226 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:20.848232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:20.848326 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:20.851022 (+ 2696us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:20.851202 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:20.852393 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:20.858051 (+ 5658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:20.858869 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:20.871336 (+ 12467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:20.871646 (+ 310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:20.871654 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:20.871659 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:20.871696 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:20.876410 (+ 4714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:20.877583 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:20.877589 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:20.877593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:20.877761 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:20.878438 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:20.878444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:20.878447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:20.878827 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:20.878961 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:20.882664 (+ 3703us) server_negotiation.cc:299] Negotiation successful
0808 03:00:20.882925 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30478,"thread_start_us":30986,"threads_started":1}
I20260808 03:00:20.885181 19126 heartbeater.cc:347] Connected to a master server at 127.0.205.190:41417
I20260808 03:00:20.885533 19126 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:20.886528 19126 heartbeater.cc:511] Master 127.0.205.190:41417 requested a full tablet report, sending...
I20260808 03:00:20.889796 18802 ts_manager.cc:194] Registered new tserver with Master: a9ea99186d2d4b9aac3a13e2809a2e01 (127.0.205.129:43859)
I20260808 03:00:20.891999 18802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:50595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
W20260808 03:00:20.938090 822 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.205.130: real 0.118s user 0.001s sys 0.024s
I20260808 03:00:20.939144 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:42885/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:41417
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:21.336769 19134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:21.337090 19134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:21.337270 19134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:21.345062 19134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:21.345185 19134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:21.345301 19134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:00:21.355614 19134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.205.148:42885/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:41417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:21.357002 19134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:21.358436 19134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:21.372469 19152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:21.372570 19157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:21.372548 19154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:21.373384 19134 server_base.cc:1034] running on GCE node
I20260808 03:00:21.374221 19134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:21.375315 19134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:21.376533 19134 hybrid_clock.cc:648] HybridClock initialized: now 1786158021376485 us; error 37 us; skew 500 ppm
I20260808 03:00:21.376881 19134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:21 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158021, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:21.380653 19134 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:00:21.382381 19134 webserver.cc:460] Webserver started at http://127.0.205.130:44243/ using document root <none> and password file <none>
I20260808 03:00:21.383193 19134 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:21.383380 19134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:21.383772 19134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:00:21 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158021, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 03:00:21.402269 19134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "1630d9bbad7a43b38d982556b81482c3"
format_stamp: "Formatted at 2026-08-08 03:00:21 on dist-test-slave-6ntk"
server_key: "ebe1e4bf83073b874f651482520b5dc4"
server_key_iv: "c766df2aa899f478e6a0939743219cd9"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:21.403183 19134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "1630d9bbad7a43b38d982556b81482c3"
format_stamp: "Formatted at 2026-08-08 03:00:21 on dist-test-slave-6ntk"
server_key: "ebe1e4bf83073b874f651482520b5dc4"
server_key_iv: "c766df2aa899f478e6a0939743219cd9"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:21.409137 19134 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260808 03:00:21.427510 19167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:21.429423 19134 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:00:21.429632 19134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "1630d9bbad7a43b38d982556b81482c3"
format_stamp: "Formatted at 2026-08-08 03:00:21 on dist-test-slave-6ntk"
server_key: "ebe1e4bf83073b874f651482520b5dc4"
server_key_iv: "c766df2aa899f478e6a0939743219cd9"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:21.429963 19134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:21.449843 19134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:21.453775 19134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:21.454255 19134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:21.455297 19134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:21.456732 19134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:21.456805 19134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:21.456902 19134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:21.456938 19134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:21.494168 19134 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:39557
I20260808 03:00:21.494205 19334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:39557 every 8 connection(s)
I20260808 03:00:21.495570 19134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:00:21.496922 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 19134
I20260808 03:00:21.497115 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 03:00:21.497689 822 external_mini_cluster.cc:1428] Setting key c1cbce95a92d11ad654f3ea8782177ee
Aug 08 03:00:21 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158021, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:21.518388 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:21.498577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:51969 (local address 127.0.205.190:41417)
0808 03:00:21.499173 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:21.499180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:21.500339 (+ 1159us) server_negotiation.cc:407] Connection header received
0808 03:00:21.502267 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:21.502273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:21.502381 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:21.502506 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:21.504003 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:21.504798 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:21.506547 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:21.506978 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:21.507130 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:21.510775 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:21.510780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:21.510784 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:21.510821 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:21.514607 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:21.515463 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:21.515472 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:21.515478 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:21.515644 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:21.516127 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:21.516136 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:21.516141 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:21.516587 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:21.516752 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:21.517548 (+ 796us) server_negotiation.cc:299] Negotiation successful
0808 03:00:21.517840 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":315,"threads_started":1}
I20260808 03:00:21.519325 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:21.499273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41417 (local address 127.0.205.130:51969)
0808 03:00:21.500167 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:21.500207 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:21.501993 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:21.502770 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:21.502779 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:21.503227 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:21.503813 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:21.503842 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:21.504976 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:21.504980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:21.506388 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:21.506409 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:21.507231 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:21.507239 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:21.507492 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:21.508228 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:21.508252 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:21.510492 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:21.514783 (+ 4291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:21.514791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:21.514808 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:21.515301 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:21.515758 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:21.515761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:21.515763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:21.515991 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:21.516729 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:21.516735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:21.517111 (+ 376us) client_negotiation.cc:769] Sending connection context
0808 03:00:21.517360 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 03:00:21.517788 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":351,"threads_started":1}
I20260808 03:00:21.520977 19335 heartbeater.cc:347] Connected to a master server at 127.0.205.190:41417
I20260808 03:00:21.521347 19335 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:21.522396 19335 heartbeater.cc:511] Master 127.0.205.190:41417 requested a full tablet report, sending...
I20260808 03:00:21.524190 18802 ts_manager.cc:194] Registered new tserver with Master: 1630d9bbad7a43b38d982556b81482c3 (127.0.205.130:39557)
I20260808 03:00:21.525043 18802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:51969
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 03:00:21.570545 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:42885/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:41417
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:00:21.896342 19126 heartbeater.cc:503] Master 127.0.205.190:41417 was elected leader, sending a full tablet report...
W20260808 03:00:21.927284 19344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:21.927649 19344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:21.927832 19344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:21.935297 19344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:21.935482 19344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:21.935678 19344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:00:21.945943 19344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.205.148:42885/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:41417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:21.947464 19344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:21.948802 19344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:21.962525 19352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:21.962627 19353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:21.962641 19357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:21.963413 19344 server_base.cc:1034] running on GCE node
I20260808 03:00:21.964084 19344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:21.965209 19344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:21.966454 19344 hybrid_clock.cc:648] HybridClock initialized: now 1786158021966398 us; error 51 us; skew 500 ppm
I20260808 03:00:21.966806 19344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:21 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158021, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:21.970435 19344 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:00:21.972062 19344 webserver.cc:460] Webserver started at http://127.0.205.131:44865/ using document root <none> and password file <none>
I20260808 03:00:21.972867 19344 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:21.973028 19344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:21.973379 19344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 03:00:21 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158021, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 03:00:21.991700 19344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "7fc1c60f021c48b299e1ae567b6a3c2c"
format_stamp: "Formatted at 2026-08-08 03:00:21 on dist-test-slave-6ntk"
server_key: "ec7a1f6e6dc2a4c87222513b3582f155"
server_key_iv: "5bfd8fe1e7e1f74ab5a812877697b1f7"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:21.992910 19344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "7fc1c60f021c48b299e1ae567b6a3c2c"
format_stamp: "Formatted at 2026-08-08 03:00:21 on dist-test-slave-6ntk"
server_key: "ec7a1f6e6dc2a4c87222513b3582f155"
server_key_iv: "5bfd8fe1e7e1f74ab5a812877697b1f7"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:22.000406 19344 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260808 03:00:22.018601 19368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:22.020530 19344 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260808 03:00:22.020749 19344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "7fc1c60f021c48b299e1ae567b6a3c2c"
format_stamp: "Formatted at 2026-08-08 03:00:21 on dist-test-slave-6ntk"
server_key: "ec7a1f6e6dc2a4c87222513b3582f155"
server_key_iv: "5bfd8fe1e7e1f74ab5a812877697b1f7"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:22.021042 19344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:22.038232 19344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:22.042913 19344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:22.043255 19344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:22.044277 19344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:22.045795 19344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:22.045897 19344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:22.046043 19344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:22.046114 19344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:22.082343 19344 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:43713
I20260808 03:00:22.082387 19542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:43713 every 8 connection(s)
I20260808 03:00:22.083845 19344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:00:22.092489 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 19344
I20260808 03:00:22.092727 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 03:00:22.093287 822 external_mini_cluster.cc:1428] Setting key c650354447e88ee258087b111fa8db7f
Aug 08 03:00:22 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158021, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:22.104723 19549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:22.086716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:42993 (local address 127.0.205.190:41417)
0808 03:00:22.087245 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:22.087250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:22.088046 (+ 796us) server_negotiation.cc:407] Connection header received
0808 03:00:22.090053 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:22.090059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:22.090172 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:22.090298 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:22.091805 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:22.092626 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:22.094565 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:22.095219 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:22.095360 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:22.099094 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:22.099099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:22.099102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:22.099136 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:22.101776 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:22.102649 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:22.102653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:22.102656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:22.102787 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:22.103188 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:22.103191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:22.103193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:22.103465 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:22.103586 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:22.104286 (+ 700us) server_negotiation.cc:299] Negotiation successful
0808 03:00:22.104465 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":292,"threads_started":1}
I20260808 03:00:22.105774 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:22.087146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41417 (local address 127.0.205.131:42993)
0808 03:00:22.087880 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:22.087920 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:22.089711 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:22.090567 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:22.090576 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:22.091041 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:22.091630 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:22.091646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:22.092789 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:22.092812 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:22.094414 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:22.094444 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:22.095474 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:22.095486 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:22.095779 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:22.096583 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:22.096619 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:22.098887 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:22.102015 (+ 3128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:22.102022 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:22.102035 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:22.102527 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:22.102888 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:22.102891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:22.102893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:22.103083 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:22.103587 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:22.103593 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:22.103943 (+ 350us) client_negotiation.cc:769] Sending connection context
0808 03:00:22.104178 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 03:00:22.104584 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":302,"threads_started":1}
I20260808 03:00:22.107443 19545 heartbeater.cc:347] Connected to a master server at 127.0.205.190:41417
I20260808 03:00:22.107836 19545 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:22.108804 19545 heartbeater.cc:511] Master 127.0.205.190:41417 requested a full tablet report, sending...
I20260808 03:00:22.110380 18802 ts_manager.cc:194] Registered new tserver with Master: 7fc1c60f021c48b299e1ae567b6a3c2c (127.0.205.131:43713)
I20260808 03:00:22.111259 18802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:42993
I20260808 03:00:22.122048 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:00:22.123855 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 18925
Aug 08 03:00:22 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 03:00:22.141610 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:22.124874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41910 (local address 127.0.205.129:43859)
0808 03:00:22.125604 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:22.125611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:22.125639 (+ 28us) server_negotiation.cc:407] Connection header received
0808 03:00:22.125699 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:22.125705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:22.126134 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:22.126275 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:22.127732 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:22.128924 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:22.130329 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:22.130919 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:22.131026 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:22.135520 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:22.135533 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:22.135548 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:22.135583 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:22.138766 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:22.139464 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:22.139471 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:22.139474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:22.139618 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:22.140057 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:22.140060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:22.140062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:22.140566 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:22.140814 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:22.141071 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 03:00:22.141289 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":407,"threads_started":1}
I20260808 03:00:22.142975 19117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:22.234366 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 18925
I20260808 03:00:22.259418 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 19134
Aug 08 03:00:22 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 03:00:22.282300 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:22.260484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45156 (local address 127.0.205.130:39557)
0808 03:00:22.261125 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:22.261133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:22.261160 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:00:22.261227 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:22.261233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:22.261470 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:22.261612 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:22.263260 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:22.264300 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:22.265931 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:22.266681 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:22.266803 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:22.276093 (+ 9290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:22.276111 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:22.276129 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:22.276188 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:22.279316 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:22.280064 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:22.280071 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:22.280074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:22.280219 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:22.280580 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:22.280583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:22.280585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:22.280977 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:22.281218 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:22.281476 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 03:00:22.281704 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":332,"threads_started":1}
I20260808 03:00:22.283742 19327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:22.373409 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 19134
I20260808 03:00:22.399360 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 19344
Aug 08 03:00:22 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 03:00:22.417451 19562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:22.400462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.0.205.131:43713)
0808 03:00:22.401132 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:22.401138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:22.401168 (+ 30us) server_negotiation.cc:407] Connection header received
0808 03:00:22.401232 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:22.401239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:22.401486 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:22.401615 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:22.403035 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:22.404107 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:22.405407 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:22.406152 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:22.406255 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:22.410959 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:22.410974 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:22.410991 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:22.411033 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:22.414328 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:22.415068 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:22.415076 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:22.415079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:22.415242 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:22.415668 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:22.415672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:22.415674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:22.416100 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:22.416375 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:22.416770 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 03:00:22.417002 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":355,"threads_started":1}
I20260808 03:00:22.419049 19536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:22.510108 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 19344
I20260808 03:00:22.541584 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 18757
I20260808 03:00:22.543125 18846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:22.655222 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 18757
I20260808 03:00:22.687878 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:41417
--webserver_interface=127.0.205.190
--webserver_port=36699
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:41417
--ranger_config_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:42885/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.205.148:42885/kms with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:23.034523 19566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:23.034905 19566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:23.035077 19566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:23.042562 19566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:00:23.042721 19566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:23.042834 19566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:23.042939 19566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:00:23.043033 19566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:00:23.053610 19566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.205.148:42885/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:41417
--ranger_config_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:41417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=36699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:23.055147 19566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:23.056584 19566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:23.070313 19578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:23.070362 19576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:23.070386 19575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:23.071391 19566 server_base.cc:1034] running on GCE node
I20260808 03:00:23.072463 19566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:23.074270 19566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:23.075544 19566 hybrid_clock.cc:648] HybridClock initialized: now 1786158023075475 us; error 57 us; skew 500 ppm
I20260808 03:00:23.075973 19566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:23 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158023, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:23.079536 19566 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:00:23.081301 19566 webserver.cc:460] Webserver started at http://127.0.205.190:36699/ using document root <none> and password file <none>
I20260808 03:00:23.082137 19566 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:23.082305 19566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 03:00:23 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158023, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 03:00:23.168967 19566 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:00:23.173027 19589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:23.174957 19566 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260808 03:00:23.175186 19566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "1affc4a6109a4af5abfb9bdac6e57da7"
format_stamp: "Formatted at 2026-08-08 03:00:18 on dist-test-slave-6ntk"
server_key: "0550a1e74efaac233b867dcdad5b01b9"
server_key_iv: "64f9e0e003dd45b0b215bfef0a6adbbf"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:23.176369 19566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:23.202080 19566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:23.206470 19566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:23.206724 19566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:23.226755 19566 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:41417
I20260808 03:00:23.226768 19669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:41417 every 8 connection(s)
I20260808 03:00:23.228503 19566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:00:23.235728 19670 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:00:23.238639 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 19566
I20260808 03:00:23.239246 19670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: Bootstrap starting.
I20260808 03:00:23.239599 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:42885/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:43859
--local_ip_for_outbound_sockets=127.0.205.129
--tserver_master_addrs=127.0.205.190:41417
--webserver_port=38085
--webserver_interface=127.0.205.129
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.205.148:42885/kms with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:00:23.259101 19670 log.cc:824] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:23.268925 19670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:00:23.269560 19670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: Bootstrap complete.
I20260808 03:00:23.279119 19670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } }
I20260808 03:00:23.280072 19670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1affc4a6109a4af5abfb9bdac6e57da7, State: Initialized, Role: FOLLOWER
I20260808 03:00:23.280776 19670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } }
I20260808 03:00:23.281036 19670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:00:23.281206 19670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:00:23.281457 19670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 1 FOLLOWER]: Advancing to term 2
I20260808 03:00:23.285523 19670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } }
I20260808 03:00:23.286170 19670 leader_election.cc:302] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1affc4a6109a4af5abfb9bdac6e57da7; no voters:
I20260808 03:00:23.286850 19670 leader_election.cc:288] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 03:00:23.286991 19674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 03:00:23.288157 19674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [term 2 LEADER]: Becoming Leader. State: Replica: 1affc4a6109a4af5abfb9bdac6e57da7, State: Running, Role: LEADER
I20260808 03:00:23.288757 19674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } }
I20260808 03:00:23.289273 19670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:00:23.291623 19675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } } }
I20260808 03:00:23.292088 19675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:23.291807 19676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1affc4a6109a4af5abfb9bdac6e57da7. Latest consensus state: current_term: 2 leader_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affc4a6109a4af5abfb9bdac6e57da7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 41417 } } }
I20260808 03:00:23.292222 19676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:23.293300 19682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:00:23.312901 19682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:00:23.315657 19682 catalog_manager.cc:1259] Loaded cluster ID: 19e286a6f89a47209f08812602962831
I20260808 03:00:23.315874 19682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:00:23.319926 19682 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:00:23.321621 19682 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: Loaded TSK: 0
I20260808 03:00:23.323278 19682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:00:23.326941 19670 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260808 03:00:23.722607 19672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:23.723057 19672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:23.723260 19672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:23.736382 19672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:23.736562 19672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:23.736789 19672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:00:23.754253 19672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.205.148:42885/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:43859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=38085
--tserver_master_addrs=127.0.205.190:41417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:23.756256 19672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:23.758307 19672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:23.802501 19726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:23.802488 19727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:23.820976 19672 server_base.cc:1034] running on GCE node
W20260808 03:00:23.821820 19729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:23.822804 19672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:23.881342 19672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:23.886006 19672 hybrid_clock.cc:648] HybridClock initialized: now 1786158023885899 us; error 114 us; skew 500 ppm
I20260808 03:00:23.886525 19672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:23 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158023, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:23.895874 19672 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:00:23.899462 19672 webserver.cc:460] Webserver started at http://127.0.205.129:38085/ using document root <none> and password file <none>
I20260808 03:00:23.900300 19672 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:23.900490 19672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 03:00:23 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158023, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 03:00:23.939572 19672 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.005s sys 0.005s
I20260808 03:00:23.959461 19741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:23.961730 19672 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.007s sys 0.012s
I20260808 03:00:23.961956 19672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "a9ea99186d2d4b9aac3a13e2809a2e01"
format_stamp: "Formatted at 2026-08-08 03:00:19 on dist-test-slave-6ntk"
server_key: "53a942e276bc943613371648397e6d8c"
server_key_iv: "b4bb0ae071190fcea470791729882e46"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:23.963478 19672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:23.994246 19672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:24.000375 19672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:24.000742 19672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:24.002316 19672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:24.004302 19672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:24.004424 19672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:24.004541 19672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:24.004630 19672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:24.095430 19672 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:43859
I20260808 03:00:24.095475 19914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:43859 every 8 connection(s)
I20260808 03:00:24.097044 19672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:00:24.106717 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 19672
I20260808 03:00:24.107601 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:42885/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:39557
--local_ip_for_outbound_sockets=127.0.205.130
--tserver_master_addrs=127.0.205.190:41417
--webserver_port=44243
--webserver_interface=127.0.205.130
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.205.148:42885/kms with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:00:24 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158023, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:24.131291 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:24.100490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:54157 (local address 127.0.205.190:41417)
0808 03:00:24.106488 (+ 5998us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:24.106511 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:24.106567 (+ 56us) server_negotiation.cc:407] Connection header received
0808 03:00:24.107123 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:24.107146 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:24.107732 (+ 586us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:24.108207 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:24.110419 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:24.112516 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:24.115524 (+ 3008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:24.116662 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:24.116883 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:24.121439 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:24.121451 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:24.121473 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:24.121526 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:24.125802 (+ 4276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:24.126962 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:24.126972 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:24.126982 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:24.127159 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:24.127640 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:24.127645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:24.127648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:24.128193 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:24.128416 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:24.129252 (+ 836us) server_negotiation.cc:299] Negotiation successful
0808 03:00:24.129640 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5669,"thread_start_us":6100,"threads_started":1}
I20260808 03:00:24.131363 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:24.100506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41417 (local address 127.0.205.129:54157)
0808 03:00:24.102098 (+ 1592us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:24.102173 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:24.104982 (+ 2809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:24.108440 (+ 3458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:24.108453 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:24.109196 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:24.110148 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:24.110178 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:24.112762 (+ 2584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:24.112769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:24.115306 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:24.115335 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:24.116963 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:24.116971 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:24.117261 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:24.118200 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:24.118234 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:24.121092 (+ 2858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:24.126082 (+ 4990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:24.126093 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:24.126111 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:24.126785 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:24.127296 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:24.127302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:24.127305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:24.127506 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:24.128351 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:24.128360 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:24.128717 (+ 357us) client_negotiation.cc:769] Sending connection context
0808 03:00:24.128998 (+ 281us) client_negotiation.cc:240] Negotiation successful
0808 03:00:24.129622 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1226,"thread_start_us":986,"threads_started":1}
I20260808 03:00:24.135346 19915 heartbeater.cc:347] Connected to a master server at 127.0.205.190:41417
I20260808 03:00:24.135742 19915 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:24.136574 19915 heartbeater.cc:511] Master 127.0.205.190:41417 requested a full tablet report, sending...
W20260808 03:00:24.140072 19915 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:41417 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 03:00:24.140362 19915 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:24.140720 19915 heartbeater.cc:511] Master 127.0.205.190:41417 requested a full tablet report, sending...
I20260808 03:00:24.142225 19915 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:24.142550 19915 heartbeater.cc:511] Master 127.0.205.190:41417 requested a full tablet report, sending...
W20260808 03:00:24.143904 19915 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 03:00:24.622586 19920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:24.623066 19920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:24.623278 19920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:24.636816 19920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:24.637008 19920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:24.637291 19920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:00:24.655905 19920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.205.148:42885/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:39557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=44243
--tserver_master_addrs=127.0.205.190:41417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:24.657995 19920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:24.660048 19920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:24.680333 19944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:24.681810 19943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:24.681972 19946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:24.683718 19920 server_base.cc:1034] running on GCE node
I20260808 03:00:24.684674 19920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:24.686184 19920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:24.687500 19920 hybrid_clock.cc:648] HybridClock initialized: now 1786158024687429 us; error 49 us; skew 500 ppm
I20260808 03:00:24.687968 19920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:24 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:24.692979 19920 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:00:24.695274 19920 webserver.cc:460] Webserver started at http://127.0.205.130:44243/ using document root <none> and password file <none>
I20260808 03:00:24.696473 19920 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:24.696676 19920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 03:00:24 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
Aug 08 03:00:24 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786158024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:24.734598 19920 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.003s sys 0.005s
I20260808 03:00:24.744891 19958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:24.747308 19920 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.009s sys 0.001s
I20260808 03:00:24.747562 19920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "1630d9bbad7a43b38d982556b81482c3"
format_stamp: "Formatted at 2026-08-08 03:00:21 on dist-test-slave-6ntk"
server_key: "ebe1e4bf83073b874f651482520b5dc4"
server_key_iv: "c766df2aa899f478e6a0939743219cd9"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:24.749239 19920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:24.786226 19920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:24.791114 19920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:24.791432 19920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:24.792578 19920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:24.794682 19920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:24.794804 19920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:24.794934 19920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:24.795023 19920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:24.851830 19920 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:39557
I20260808 03:00:24.851881 20125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:39557 every 8 connection(s)
I20260808 03:00:24.853354 19920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:00:24.857859 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 19920
I20260808 03:00:24.858747 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.205.148:42885/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:43713
--local_ip_for_outbound_sockets=127.0.205.131
--tserver_master_addrs=127.0.205.190:41417
--webserver_port=44865
--webserver_interface=127.0.205.131
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.205.148:42885/kms with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:00:24 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:24.893407 20128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:24.856887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:35053 (local address 127.0.205.190:41417)
0808 03:00:24.857438 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:24.857448 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:24.858787 (+ 1339us) server_negotiation.cc:407] Connection header received
0808 03:00:24.862217 (+ 3430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:24.862225 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:24.862322 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:24.862471 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:24.864672 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:24.865845 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:24.868492 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:24.869139 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:24.869251 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:24.882687 (+ 13436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:24.882698 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:24.882706 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:24.882759 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:24.887242 (+ 4483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:24.889650 (+ 2408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:24.889657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:24.889662 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:24.889847 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:24.890598 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:24.890606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:24.890612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:24.891098 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:24.891253 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:24.892579 (+ 1326us) server_negotiation.cc:299] Negotiation successful
0808 03:00:24.892877 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":268,"threads_started":1}
I20260808 03:00:24.901402 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:24.857526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41417 (local address 127.0.205.130:35053)
0808 03:00:24.858578 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:24.858643 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:24.861809 (+ 3166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:24.862769 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:24.862784 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:24.863468 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:24.864431 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:24.864462 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:24.866090 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:24.866097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:24.868290 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:24.868321 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:24.869618 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:24.869627 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:24.870059 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:24.878174 (+ 8115us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:24.878216 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:24.881835 (+ 3619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:24.888589 (+ 6754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:24.888605 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:24.888628 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:24.889446 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:24.890082 (+ 636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:24.890088 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:24.890095 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:24.890410 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:24.891384 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:24.891394 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:24.892007 (+ 613us) client_negotiation.cc:769] Sending connection context
0808 03:00:24.893924 (+ 1917us) client_negotiation.cc:240] Negotiation successful
0808 03:00:24.894641 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":486,"threads_started":1}
I20260808 03:00:24.903599 20126 heartbeater.cc:347] Connected to a master server at 127.0.205.190:41417
I20260808 03:00:24.903998 20126 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:24.904996 20126 heartbeater.cc:511] Master 127.0.205.190:41417 requested a full tablet report, sending...
I20260808 03:00:24.907603 19612 ts_manager.cc:194] Registered new tserver with Master: 1630d9bbad7a43b38d982556b81482c3 (127.0.205.130:39557)
I20260808 03:00:24.909602 19612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:35053
I20260808 03:00:25.144076 19915 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:25.144596 19915 heartbeater.cc:511] Master 127.0.205.190:41417 requested a full tablet report, sending...
I20260808 03:00:25.146147 19612 ts_manager.cc:194] Registered new tserver with Master: a9ea99186d2d4b9aac3a13e2809a2e01 (127.0.205.129:43859)
I20260808 03:00:25.147501 19612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:54157
Aug 08 03:00:25 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
W20260808 03:00:25.341128 20130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:25.341491 20130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:25.341660 20130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:25.350706 20130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:25.350893 20130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:25.351063 20130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:00:25.366279 20130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.205.148:42885/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:43713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=44865
--tserver_master_addrs=127.0.205.190:41417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:25.368252 20130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:25.370278 20130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:25.389467 20141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:25.396055 20145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:25.396289 20140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:25.397168 20130 server_base.cc:1034] running on GCE node
I20260808 03:00:25.397981 20130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:25.399250 20130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:25.400560 20130 hybrid_clock.cc:648] HybridClock initialized: now 1786158025400494 us; error 71 us; skew 500 ppm
I20260808 03:00:25.400931 20130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:25 dist-test-slave-6ntk krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158025, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:25.404553 20130 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:00:25.406656 20130 webserver.cc:460] Webserver started at http://127.0.205.131:44865/ using document root <none> and password file <none>
I20260808 03:00:25.407480 20130 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:25.407665 20130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 03:00:25 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158025, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
I20260808 03:00:25.431317 20130 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 03:00:25.434558 20157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:25.436295 20130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:00:25.436497 20130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "7fc1c60f021c48b299e1ae567b6a3c2c"
format_stamp: "Formatted at 2026-08-08 03:00:21 on dist-test-slave-6ntk"
server_key: "ec7a1f6e6dc2a4c87222513b3582f155"
server_key_iv: "5bfd8fe1e7e1f74ab5a812877697b1f7"
server_key_version: "kuduclusterkey@0"
I20260808 03:00:25.437606 20130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:25.459139 20130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:25.463846 20130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:25.464182 20130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:25.465507 20130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:25.467411 20130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:25.467545 20130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:25.467698 20130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:25.467794 20130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:25.529130 20130 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:43713
I20260808 03:00:25.529191 20333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:43713 every 8 connection(s)
I20260808 03:00:25.531041 20130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:00:25.540534 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 20130
Aug 08 03:00:25 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for HTTP/127.0.205.148@KRBTEST.COM
Aug 08 03:00:25 dist-test-slave-6ntk krb5kdc[18049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158025, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:25.590545 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.542804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44118 (local address 127.0.205.190:41417)
0808 03:00:25.546736 (+ 3932us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:25.546741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:25.557995 (+ 11254us) server_negotiation.cc:407] Connection header received
0808 03:00:25.558267 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:25.558274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:25.558398 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:25.558528 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:25.565316 (+ 6788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.566683 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.571507 (+ 4824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.572208 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.572335 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.577158 (+ 4823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:25.577165 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:25.577170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:25.577215 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:25.581013 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:25.582938 (+ 1925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:25.582944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:25.582948 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:25.584384 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:25.586435 (+ 2051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:25.586443 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:25.586447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:25.586856 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:25.587166 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:25.589805 (+ 2639us) server_negotiation.cc:299] Negotiation successful
0808 03:00:25.590093 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":3804,"thread_start_us":3661,"threads_started":1}
I20260808 03:00:25.592123 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.542203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:35915 (local address 127.0.205.190:41417)
0808 03:00:25.546306 (+ 4103us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:25.546313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:25.559101 (+ 12788us) server_negotiation.cc:407] Connection header received
0808 03:00:25.565442 (+ 6341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:25.565448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:25.565572 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:25.565679 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:25.568085 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.569184 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.571449 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.572715 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.572846 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.582134 (+ 9288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:25.582143 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:25.582147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:25.582190 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:25.586275 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:25.588280 (+ 2005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:25.588287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:25.588293 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:25.588460 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:25.589107 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:25.589113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:25.589116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:25.589507 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:25.589649 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:25.591433 (+ 1784us) server_negotiation.cc:299] Negotiation successful
0808 03:00:25.591694 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":521,"server-negotiator.queue_time_us":3961,"thread_start_us":3808,"threads_started":1}
I20260808 03:00:25.593514 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.541324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:41417 (local address 127.0.205.131:35915)
0808 03:00:25.558884 (+ 17560us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:25.558962 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:25.561596 (+ 2634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:25.566381 (+ 4785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:25.566391 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:25.566923 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:25.567834 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.567863 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.569360 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.569364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.571258 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.571280 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.575887 (+ 4607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.575896 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.576269 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.578027 (+ 1758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:25.578062 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:25.580930 (+ 2868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:25.587327 (+ 6397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:25.587339 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:25.587360 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:25.588096 (+ 736us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:25.588643 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:25.588648 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:25.588652 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:25.588952 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:25.590427 (+ 1475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:25.590438 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:25.590987 (+ 549us) client_negotiation.cc:769] Sending connection context
0808 03:00:25.591310 (+ 323us) client_negotiation.cc:240] Negotiation successful
0808 03:00:25.591995 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17167,"thread_start_us":20416,"threads_started":1}
I20260808 03:00:25.595425 20340 heartbeater.cc:347] Connected to a master server at 127.0.205.190:41417
I20260808 03:00:25.595777 20340 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:25.596670 20340 heartbeater.cc:511] Master 127.0.205.190:41417 requested a full tablet report, sending...
I20260808 03:00:25.598901 19610 ts_manager.cc:194] Registered new tserver with Master: 7fc1c60f021c48b299e1ae567b6a3c2c (127.0.205.131:43713)
I20260808 03:00:25.600274 19610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:35915
I20260808 03:00:25.608167 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:00:25.680063 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.650904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.0.205.190:41417)
0808 03:00:25.651137 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:25.651144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:25.658738 (+ 7594us) server_negotiation.cc:407] Connection header received
0808 03:00:25.658969 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:25.658975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:25.659086 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:25.659229 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:25.664284 (+ 5055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.665437 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.667586 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.668269 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.668386 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.672368 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:25.672376 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:25.672381 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:25.672434 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:25.676262 (+ 3828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:25.677383 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:25.677389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:25.677393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:25.677563 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:25.678105 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:25.678110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:25.678114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:25.678531 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:25.678775 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:25.679384 (+ 609us) server_negotiation.cc:299] Negotiation successful
0808 03:00:25.679639 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:00:25.686862 19610 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:00:25.698597 19610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:00:25.767174 20365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.736975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45164 (local address 127.0.205.130:39557)
0808 03:00:25.743258 (+ 6283us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:25.743278 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:25.751626 (+ 8348us) server_negotiation.cc:407] Connection header received
0808 03:00:25.752673 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:25.752682 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:25.752928 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:25.753109 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:25.755091 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.757020 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.762735 (+ 5715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.764348 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.764475 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.764714 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:25.766604 (+ 1890us) server_negotiation.cc:299] Negotiation successful
0808 03:00:25.766815 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6082,"thread_start_us":5941,"threads_started":1}
I20260808 03:00:25.769155 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.736536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:39557 (local address 127.0.0.1:45164)
0808 03:00:25.751467 (+ 14931us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:25.751490 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:25.752441 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:25.753502 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:25.753511 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:25.753544 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:25.754009 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.754026 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.757266 (+ 3240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.757271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.759775 (+ 2504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.759792 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.764988 (+ 5196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.764994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.765960 (+ 966us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.766029 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 03:00:25.767791 (+ 1762us) client_negotiation.cc:240] Negotiation successful
0808 03:00:25.768040 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14792,"mutex_wait_us":364,"thread_start_us":6964,"threads_started":1}
I20260808 03:00:25.770690 20364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.733575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43713 (local address 127.0.0.1:47676)
0808 03:00:25.743902 (+ 10327us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:25.743959 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:25.744158 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:25.758555 (+ 14397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:25.758561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:25.758597 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:25.758979 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.758990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.763566 (+ 4576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.763572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.765646 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.765663 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.769898 (+ 4235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.769905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.770098 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.770134 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 03:00:25.770207 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 03:00:25.770349 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10056,"mutex_wait_us":24,"thread_start_us":18088,"threads_started":1}
I20260808 03:00:25.772289 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.734329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.0.205.131:43713)
0808 03:00:25.751325 (+ 16996us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:25.751335 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:25.751374 (+ 39us) server_negotiation.cc:407] Connection header received
0808 03:00:25.751495 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:25.751503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:25.751795 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:25.759252 (+ 7457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:25.759579 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.761212 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.765792 (+ 4580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.767413 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.771388 (+ 3975us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.771664 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:25.771740 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 03:00:25.771915 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16792,"thread_start_us":17558,"threads_started":1}
I20260808 03:00:25.774691 20026 tablet_service.cc:1467] Processing CreateTablet for tablet de8a8211bf7a4eaa810d5d2b4b13a022 (DEFAULT_TABLE table=test-table [id=993e8869ceab4ecfa424bdadd5d80a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:00:25.776427 20026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8a8211bf7a4eaa810d5d2b4b13a022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:25.781911 20225 tablet_service.cc:1467] Processing CreateTablet for tablet de8a8211bf7a4eaa810d5d2b4b13a022 (DEFAULT_TABLE table=test-table [id=993e8869ceab4ecfa424bdadd5d80a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:00:25.783510 20225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8a8211bf7a4eaa810d5d2b4b13a022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:25.792590 20364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.752152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43859 (local address 127.0.0.1:41926)
0808 03:00:25.770933 (+ 18781us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:25.770954 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:25.771083 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:25.773149 (+ 2066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:25.773154 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:25.773187 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:25.773416 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.773431 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.776532 (+ 3101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.776538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.778446 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.778462 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.791759 (+ 13297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.791766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.791969 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.792010 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 03:00:25.792081 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 03:00:25.792248 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18676,"thread_start_us":24035,"threads_started":1}
I20260808 03:00:25.793672 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.752407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41926 (local address 127.0.205.129:43859)
0808 03:00:25.769469 (+ 17062us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:25.769478 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:25.772518 (+ 3040us) server_negotiation.cc:407] Connection header received
0808 03:00:25.772644 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:25.772652 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:25.772910 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:25.773985 (+ 1075us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:25.774256 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.775810 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.778693 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.791524 (+ 12831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.792839 (+ 1315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.793098 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:25.793185 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 03:00:25.793366 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16854,"thread_start_us":17065,"threads_started":1}
I20260808 03:00:25.797740 19808 tablet_service.cc:1467] Processing CreateTablet for tablet de8a8211bf7a4eaa810d5d2b4b13a022 (DEFAULT_TABLE table=test-table [id=993e8869ceab4ecfa424bdadd5d80a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:00:25.799793 19808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8a8211bf7a4eaa810d5d2b4b13a022. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:25.811385 20369 tablet_bootstrap.cc:492] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: Bootstrap starting.
I20260808 03:00:25.817638 20369 tablet_bootstrap.cc:654] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:25.819154 20369 log.cc:824] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:25.828900 20369 tablet_bootstrap.cc:492] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: No bootstrap required, opened a new log
I20260808 03:00:25.829233 20369 ts_tablet_manager.cc:1397] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.011s
I20260808 03:00:25.838727 20371 tablet_bootstrap.cc:492] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: Bootstrap starting.
I20260808 03:00:25.840574 20369 raft_consensus.cc:348] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 } } peers { permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 } } peers { permanent_uuid: "1630d9bbad7a43b38d982556b81482c3" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39557 } }
I20260808 03:00:25.841188 20369 raft_consensus.cc:374] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:25.841316 20369 raft_consensus.cc:729] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1630d9bbad7a43b38d982556b81482c3, State: Initialized, Role: FOLLOWER
I20260808 03:00:25.842170 20369 consensus_queue.cc:260] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 } } peers { permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 } } peers { permanent_uuid: "1630d9bbad7a43b38d982556b81482c3" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39557 } }
I20260808 03:00:25.847599 20371 tablet_bootstrap.cc:654] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:25.841857 20372 tablet_bootstrap.cc:492] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: Bootstrap starting.
I20260808 03:00:25.851913 20372 tablet_bootstrap.cc:654] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:25.853602 20371 log.cc:824] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:25.855536 20372 log.cc:824] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:25.868968 20369 ts_tablet_manager.cc:1428] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: Time spent starting tablet: real 0.040s user 0.014s sys 0.009s
I20260808 03:00:25.876513 20126 heartbeater.cc:503] Master 127.0.205.190:41417 was elected leader, sending a full tablet report...
I20260808 03:00:25.882639 20374 raft_consensus.cc:482] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:00:25.883056 20374 raft_consensus.cc:504] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 } } peers { permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 } } peers { permanent_uuid: "1630d9bbad7a43b38d982556b81482c3" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39557 } }
I20260808 03:00:25.913653 20371 tablet_bootstrap.cc:492] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: No bootstrap required, opened a new log
I20260808 03:00:25.914129 20371 ts_tablet_manager.cc:1397] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: Time spent bootstrapping tablet: real 0.076s user 0.018s sys 0.037s
I20260808 03:00:25.916229 20372 tablet_bootstrap.cc:492] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: No bootstrap required, opened a new log
I20260808 03:00:25.916611 20372 ts_tablet_manager.cc:1397] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: Time spent bootstrapping tablet: real 0.075s user 0.018s sys 0.037s
I20260808 03:00:25.916951 20374 leader_election.cc:288] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fc1c60f021c48b299e1ae567b6a3c2c (127.0.205.131:43713), a9ea99186d2d4b9aac3a13e2809a2e01 (127.0.205.129:43859)
I20260808 03:00:25.925853 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.886596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:38345 (local address 127.0.205.131:43713)
0808 03:00:25.886876 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:25.886887 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:25.896985 (+ 10098us) server_negotiation.cc:407] Connection header received
0808 03:00:25.897583 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:25.897592 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:25.897700 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:25.897796 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:25.898790 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.900290 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.913458 (+ 13168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.914939 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.915087 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.915161 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:25.925374 (+ 10213us) server_negotiation.cc:299] Negotiation successful
0808 03:00:25.925567 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:00:25.926846 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.885895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:43713 (local address 127.0.205.130:38345)
0808 03:00:25.896812 (+ 10917us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:25.896844 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:25.897362 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:25.898105 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:25.898112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:25.898149 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:25.898571 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.898586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.900550 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.900558 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.907781 (+ 7223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.913208 (+ 5427us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.921738 (+ 8530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.921749 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.922013 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.922059 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 03:00:25.926257 (+ 4198us) client_negotiation.cc:240] Negotiation successful
0808 03:00:25.926441 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10679,"thread_start_us":11097,"threads_started":1}
I20260808 03:00:25.927201 20371 raft_consensus.cc:348] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 } } peers { permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 } } peers { permanent_uuid: "1630d9bbad7a43b38d982556b81482c3" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39557 } }
I20260808 03:00:25.927832 20371 raft_consensus.cc:374] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:25.927947 20371 raft_consensus.cc:729] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fc1c60f021c48b299e1ae567b6a3c2c, State: Initialized, Role: FOLLOWER
I20260808 03:00:25.928196 20372 raft_consensus.cc:348] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 } } peers { permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 } } peers { permanent_uuid: "1630d9bbad7a43b38d982556b81482c3" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39557 } }
I20260808 03:00:25.928704 20372 raft_consensus.cc:374] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:25.928864 20372 raft_consensus.cc:729] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ea99186d2d4b9aac3a13e2809a2e01, State: Initialized, Role: FOLLOWER
I20260808 03:00:25.928738 20371 consensus_queue.cc:260] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 } } peers { permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 } } peers { permanent_uuid: "1630d9bbad7a43b38d982556b81482c3" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39557 } }
I20260808 03:00:25.929697 20372 consensus_queue.cc:260] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 } } peers { permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 } } peers { permanent_uuid: "1630d9bbad7a43b38d982556b81482c3" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39557 } }
I20260808 03:00:25.935289 20372 ts_tablet_manager.cc:1428] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
I20260808 03:00:25.935734 19915 heartbeater.cc:503] Master 127.0.205.190:41417 was elected leader, sending a full tablet report...
I20260808 03:00:25.941682 20340 heartbeater.cc:503] Master 127.0.205.190:41417 was elected leader, sending a full tablet report...
I20260808 03:00:25.942755 20371 ts_tablet_manager.cc:1428] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: Time spent starting tablet: real 0.028s user 0.026s sys 0.002s
I20260808 03:00:25.943711 20259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8a8211bf7a4eaa810d5d2b4b13a022" candidate_uuid: "1630d9bbad7a43b38d982556b81482c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" is_pre_election: true
I20260808 03:00:25.944294 20259 raft_consensus.cc:2454] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1630d9bbad7a43b38d982556b81482c3 in term 0.
I20260808 03:00:25.945078 19961 leader_election.cc:302] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1630d9bbad7a43b38d982556b81482c3, 7fc1c60f021c48b299e1ae567b6a3c2c; no voters:
I20260808 03:00:25.946344 20374 raft_consensus.cc:2781] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:00:25.946528 20374 raft_consensus.cc:482] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:00:25.946638 20374 raft_consensus.cc:3037] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:25.948280 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.909223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:37185 (local address 127.0.205.129:43859)
0808 03:00:25.909453 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:25.909463 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:25.927220 (+ 17757us) server_negotiation.cc:407] Connection header received
0808 03:00:25.930525 (+ 3305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:25.930535 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:25.930674 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:25.930759 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:25.932081 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.933643 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.944655 (+ 11012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.946016 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.946130 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.946185 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:25.947855 (+ 1670us) server_negotiation.cc:299] Negotiation successful
0808 03:00:25.948023 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 03:00:25.948498 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.908564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43859 (local address 127.0.205.130:37185)
0808 03:00:25.927104 (+ 18540us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:25.927125 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:25.929984 (+ 2859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:25.931048 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:25.931056 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:25.931095 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:25.931366 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.931381 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.934582 (+ 3201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.934591 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.937212 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:25.937234 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.947398 (+ 10164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.947405 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:25.947641 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.947682 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 03:00:25.947800 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 03:00:25.948154 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18429,"thread_start_us":33961,"threads_started":1}
I20260808 03:00:25.949785 19838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8a8211bf7a4eaa810d5d2b4b13a022" candidate_uuid: "1630d9bbad7a43b38d982556b81482c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" is_pre_election: true
I20260808 03:00:25.950018 20374 raft_consensus.cc:504] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 } } peers { permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 } } peers { permanent_uuid: "1630d9bbad7a43b38d982556b81482c3" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39557 } }
I20260808 03:00:25.950448 19838 raft_consensus.cc:2454] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1630d9bbad7a43b38d982556b81482c3 in term 0.
I20260808 03:00:25.950762 20374 leader_election.cc:288] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [CANDIDATE]: Term 1 election: Requested vote from peers 7fc1c60f021c48b299e1ae567b6a3c2c (127.0.205.131:43713), a9ea99186d2d4b9aac3a13e2809a2e01 (127.0.205.129:43859)
I20260808 03:00:25.951439 20259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8a8211bf7a4eaa810d5d2b4b13a022" candidate_uuid: "1630d9bbad7a43b38d982556b81482c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c"
I20260808 03:00:25.951490 19838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8a8211bf7a4eaa810d5d2b4b13a022" candidate_uuid: "1630d9bbad7a43b38d982556b81482c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01"
I20260808 03:00:25.951692 19838 raft_consensus.cc:3037] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:25.951707 20259 raft_consensus.cc:3037] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:25.954155 20259 raft_consensus.cc:2454] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1630d9bbad7a43b38d982556b81482c3 in term 1.
I20260808 03:00:25.954613 19961 leader_election.cc:302] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1630d9bbad7a43b38d982556b81482c3, 7fc1c60f021c48b299e1ae567b6a3c2c; no voters:
I20260808 03:00:25.954952 20374 raft_consensus.cc:2781] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:00:25.955102 19838 raft_consensus.cc:2454] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1630d9bbad7a43b38d982556b81482c3 in term 1.
I20260808 03:00:25.955229 20374 raft_consensus.cc:686] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 1 LEADER]: Becoming Leader. State: Replica: 1630d9bbad7a43b38d982556b81482c3, State: Running, Role: LEADER
I20260808 03:00:25.955819 20374 consensus_queue.cc:237] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 } } peers { permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 } } peers { permanent_uuid: "1630d9bbad7a43b38d982556b81482c3" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39557 } }
I20260808 03:00:25.960670 19610 catalog_manager.cc:5495] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1630d9bbad7a43b38d982556b81482c3 (127.0.205.130). New cstate: current_term: 1 leader_uuid: "1630d9bbad7a43b38d982556b81482c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1630d9bbad7a43b38d982556b81482c3" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 39557 } health_report { overall_health: HEALTHY } } }
I20260808 03:00:25.986896 20365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:25.981244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.0.205.130:39557)
0808 03:00:25.981482 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:25.981487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:25.981508 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:00:25.981737 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:25.981741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:25.981835 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:25.982033 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:00:25.982448 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.983179 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.984558 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:25.984975 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:25.985114 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:25.985679 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:00:25.986254 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:00:25.986426 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:25.986519 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 03:00:25.986603 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 03:00:25.995520 20259 raft_consensus.cc:1261] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c [term 1 FOLLOWER]: Refusing update from remote peer 1630d9bbad7a43b38d982556b81482c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:00:25.995520 19838 raft_consensus.cc:1261] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01 [term 1 FOLLOWER]: Refusing update from remote peer 1630d9bbad7a43b38d982556b81482c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:00:25.996512 20374 consensus_queue.cc:1035] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fc1c60f021c48b299e1ae567b6a3c2c" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 43713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:00:25.996858 20379 consensus_queue.cc:1035] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ea99186d2d4b9aac3a13e2809a2e01" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:00:26.013324 20400 mvcc.cc:205] Tried to move back new op lower bound from 7316103274467172352 to 7316103274319220736. Current Snapshot: MvccSnapshot[applied={T|T < 7316103274467172352}]
I20260808 03:00:26.028561 19612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:00:26.028805 19612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:00:26.032964 19612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1affc4a6109a4af5abfb9bdac6e57da7: Sending DeleteTablet for 3 replicas of tablet de8a8211bf7a4eaa810d5d2b4b13a022
I20260808 03:00:26.034423 20026 tablet_service.cc:1514] Processing DeleteTablet for tablet de8a8211bf7a4eaa810d5d2b4b13a022 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:00:26 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:45164
I20260808 03:00:26.034436 20225 tablet_service.cc:1514] Processing DeleteTablet for tablet de8a8211bf7a4eaa810d5d2b4b13a022 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:00:26 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:47676
I20260808 03:00:26.034510 19808 tablet_service.cc:1514] Processing DeleteTablet for tablet de8a8211bf7a4eaa810d5d2b4b13a022 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:00:26 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:41926
I20260808 03:00:26.047523 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 19672
I20260808 03:00:26.049896 20417 tablet_replica.cc:330] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: stopping tablet replica
I20260808 03:00:26.053751 20417 raft_consensus.cc:2229] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:00:26.054422 20417 raft_consensus.cc:2258] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:00:26.060599 20417 ts_tablet_manager.cc:1905] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:00:26.062692 20414 tablet_replica.cc:330] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: stopping tablet replica
I20260808 03:00:26.063227 20414 raft_consensus.cc:2229] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:00:26.063982 20414 raft_consensus.cc:2258] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:00:26.064904 20416 tablet_replica.cc:330] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: stopping tablet replica
I20260808 03:00:26.065413 20416 raft_consensus.cc:2229] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:00:26.066010 20416 raft_consensus.cc:2258] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:00:26.068240 20416 ts_tablet_manager.cc:1905] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:00:26.069026 20417 ts_tablet_manager.cc:1918] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:00:26.069264 20417 log.cc:1196] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/de8a8211bf7a4eaa810d5d2b4b13a022
I20260808 03:00:26.070116 20417 ts_tablet_manager.cc:1939] T de8a8211bf7a4eaa810d5d2b4b13a022 P a9ea99186d2d4b9aac3a13e2809a2e01: Deleting consensus metadata
I20260808 03:00:26.072508 19597 catalog_manager.cc:4841] TS a9ea99186d2d4b9aac3a13e2809a2e01 (127.0.205.129:43859): tablet de8a8211bf7a4eaa810d5d2b4b13a022 (table test-table [id=993e8869ceab4ecfa424bdadd5d80a95]) successfully deleted
I20260808 03:00:26.073068 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:26.050774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41940 (local address 127.0.205.129:43859)
0808 03:00:26.050958 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:26.050964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:26.051045 (+ 81us) server_negotiation.cc:407] Connection header received
0808 03:00:26.051285 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:26.051290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:26.051365 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:26.051483 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:26.053627 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:26.054862 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:26.057493 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:26.058184 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:26.058339 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:26.062099 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:26.062114 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:26.062133 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:26.062182 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:26.066638 (+ 4456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:26.067885 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:26.067958 (+ 73us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:26.067964 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:26.068396 (+ 432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:26.069432 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:26.069442 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:26.069446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:26.070141 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:26.070538 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:26.072177 (+ 1639us) server_negotiation.cc:299] Negotiation successful
0808 03:00:26.072479 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 03:00:26.073976 19910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:26.075840 20416 ts_tablet_manager.cc:1918] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:00:26.076038 20416 log.cc:1196] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/de8a8211bf7a4eaa810d5d2b4b13a022
I20260808 03:00:26.076767 20416 ts_tablet_manager.cc:1939] T de8a8211bf7a4eaa810d5d2b4b13a022 P 7fc1c60f021c48b299e1ae567b6a3c2c: Deleting consensus metadata
I20260808 03:00:26.077493 20414 ts_tablet_manager.cc:1905] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:00:26.078879 19597 catalog_manager.cc:4841] TS 7fc1c60f021c48b299e1ae567b6a3c2c (127.0.205.131:43713): tablet de8a8211bf7a4eaa810d5d2b4b13a022 (table test-table [id=993e8869ceab4ecfa424bdadd5d80a95]) successfully deleted
I20260808 03:00:26.088166 20414 ts_tablet_manager.cc:1918] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:00:26.088450 20414 log.cc:1196] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/de8a8211bf7a4eaa810d5d2b4b13a022
I20260808 03:00:26.089111 20414 ts_tablet_manager.cc:1939] T de8a8211bf7a4eaa810d5d2b4b13a022 P 1630d9bbad7a43b38d982556b81482c3: Deleting consensus metadata
I20260808 03:00:26.091354 19598 catalog_manager.cc:4841] TS 1630d9bbad7a43b38d982556b81482c3 (127.0.205.130:39557): tablet de8a8211bf7a4eaa810d5d2b4b13a022 (table test-table [id=993e8869ceab4ecfa424bdadd5d80a95]) successfully deleted
I20260808 03:00:26.199841 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 19672
I20260808 03:00:26.225204 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 19920
I20260808 03:00:26.240360 20365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:26.226136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45174 (local address 127.0.205.130:39557)
0808 03:00:26.226378 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:26.226385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:26.226432 (+ 47us) server_negotiation.cc:407] Connection header received
0808 03:00:26.226673 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:26.226678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:26.226751 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:26.226889 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:26.228273 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:26.229112 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:26.230541 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:26.231003 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:26.231149 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:26.233767 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:26.233780 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:26.233797 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:26.233831 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:26.237208 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:26.237954 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:26.237963 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:26.237966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:26.238170 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:26.238580 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:26.238590 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:26.238594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:26.238969 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:26.239354 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:26.239571 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 03:00:26.239956 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260808 03:00:26.241205 20120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:26.364290 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 19920
I20260808 03:00:26.391793 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 20130
I20260808 03:00:26.406118 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:26.392638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47684 (local address 127.0.205.131:43713)
0808 03:00:26.392818 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:26.392826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:26.392929 (+ 103us) server_negotiation.cc:407] Connection header received
0808 03:00:26.393152 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:26.393157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:26.393223 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:26.393342 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:26.394791 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:26.395550 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:26.396846 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:26.397266 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:26.397413 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:26.400161 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:26.400174 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:26.400192 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:26.400230 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:26.403096 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:26.403866 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:26.403880 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:26.403882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:26.404010 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:26.404418 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:26.404424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:26.404427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:26.404819 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:26.405154 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:26.405481 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 03:00:26.405723 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:00:26.406912 20326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:26.499176 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 20130
I20260808 03:00:26.524142 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 19566
I20260808 03:00:26.525466 19663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:26.640973 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 19566
I20260808 03:00:26.677399 822 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 03:00:27.022905 822 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 03:00:27.023190 822 mini_ranger.cc:66] Stopping Ranger...
I20260808 03:00:27.379446 822 mini_ranger.cc:68] Stopped Ranger
2026-08-08 03:00:27.379 UTC [18105] LOG: received smart shutdown request
2026-08-08 03:00:27.383 UTC [18105] LOG: background worker "logical replication launcher" (PID 18123) exited with exit code 1
2026-08-08 03:00:27.383 UTC [18113] LOG: shutting down
2026-08-08 03:00:27.568 UTC [18105] LOG: database system is shut down
2026-08-08T03:00:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56779 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:00:27 dist-test-slave-6ntk krb5kdc[20436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:00:27 dist-test-slave-6ntk krb5kdc[20436](info): set up 2 sockets
Aug 08 03:00:27 dist-test-slave-6ntk krb5kdc[20436](info): commencing operation
krb5kdc: starting...
W20260808 03:00:29.128747 822 mini_kdc.cc:121] Time spent starting KDC: real 1.367s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:00:29 dist-test-slave-6ntk krb5kdc[20436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:00:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:00:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:00:29.428040 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43921
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:43921,127.0.205.189:46485,127.0.205.188:37425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:29.795008 20458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:29.795444 20458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:29.795620 20458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:29.803158 20458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:00:29.803347 20458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:29.803491 20458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:29.803596 20458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:00:29.803690 20458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:00:29.814435 20458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:33833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:43921,127.0.205.189:46485,127.0.205.188:37425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:29.815909 20458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:29.817379 20458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:29.830225 20465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:29.830235 20466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:29.830637 20470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:29.831597 20458 server_base.cc:1034] running on GCE node
I20260808 03:00:29.832535 20458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:29.834192 20458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:29.835439 20458 hybrid_clock.cc:648] HybridClock initialized: now 1786158029835401 us; error 45 us; skew 500 ppm
I20260808 03:00:29.835810 20458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:29 dist-test-slave-6ntk krb5kdc[20436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:29.846900 20458 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:00:29.848601 20458 webserver.cc:460] Webserver started at http://127.0.205.190:34545/ using document root <none> and password file <none>
I20260808 03:00:29.849421 20458 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:29.849613 20458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:29.850063 20458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:00:29.854285 20458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "19897f17af3a47b985941d42ee483f03"
format_stamp: "Formatted at 2026-08-08 03:00:29 on dist-test-slave-6ntk"
server_key: "88e076d7b51af7a76bb332abfe0e400d"
server_key_iv: "2a96682b2fb16df375ff5e03152abb7d"
server_key_version: "encryptionkey@0"
I20260808 03:00:29.855187 20458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "19897f17af3a47b985941d42ee483f03"
format_stamp: "Formatted at 2026-08-08 03:00:29 on dist-test-slave-6ntk"
server_key: "88e076d7b51af7a76bb332abfe0e400d"
server_key_iv: "2a96682b2fb16df375ff5e03152abb7d"
server_key_version: "encryptionkey@0"
I20260808 03:00:29.861202 20458 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260808 03:00:29.865089 20481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:29.866708 20458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:00:29.866904 20458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "19897f17af3a47b985941d42ee483f03"
format_stamp: "Formatted at 2026-08-08 03:00:29 on dist-test-slave-6ntk"
server_key: "88e076d7b51af7a76bb332abfe0e400d"
server_key_iv: "2a96682b2fb16df375ff5e03152abb7d"
server_key_version: "encryptionkey@0"
I20260808 03:00:29.867161 20458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:29.882368 20458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:29.886906 20458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:29.887178 20458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:29.908412 20458 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:43921
I20260808 03:00:29.908471 20553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:43921 every 8 connection(s)
I20260808 03:00:29.910353 20458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:00:29.914407 20555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:29.918462 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 20458
I20260808 03:00:29.918702 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:00:29.919376 822 external_mini_cluster.cc:1428] Setting key a2ca5cfd9f30dd8d41991881d4246a27
I20260808 03:00:29.921545 20555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:00:29 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.205.189@KRBTEST.COM; defaulting to no policy
I20260808 03:00:29.949223 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:29.925108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43921 (local address 127.0.0.1:58910)
0808 03:00:29.926182 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:29.926244 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:29.929280 (+ 3036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:29.930417 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:29.930424 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:29.931292 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:29.931816 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:29.931845 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:29.933473 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:29.933482 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:29.935563 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:29.935581 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:29.936539 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:29.936549 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:29.936790 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:29.937594 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:29.937632 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:29.940922 (+ 3290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:29.944605 (+ 3683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:29.944624 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:29.944643 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:29.945324 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:29.945753 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:29.945762 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:29.945769 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:29.946054 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:29.946929 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:29.946940 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:29.947268 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 03:00:29.947571 (+ 303us) client_negotiation.cc:240] Negotiation successful
0808 03:00:29.948215 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":84,"thread_start_us":416,"threads_started":1}
I20260808 03:00:29.949242 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:29.925108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.0.205.190:43921)
0808 03:00:29.927268 (+ 2160us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:29.927274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:29.927323 (+ 49us) server_negotiation.cc:407] Connection header received
0808 03:00:29.929770 (+ 2447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:29.929777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:29.930152 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:29.930402 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:29.932079 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:29.933278 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:29.935716 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:29.936277 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:29.936461 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:29.941199 (+ 4738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:29.941208 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:29.941223 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:29.941255 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:29.944441 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:29.945471 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:29.945478 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:29.945481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:29.945629 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:29.946192 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:29.946196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:29.946198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:29.946746 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:29.946915 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:29.947696 (+ 781us) server_negotiation.cc:299] Negotiation successful
0808 03:00:29.948074 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1949,"spinlock_wait_cycles":61824,"thread_start_us":1691,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.189@KRBTEST.COM" created.
I20260808 03:00:29.952457 20555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:00:29.953577 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:29.953178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53722)
0808 03:00:29.953349 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:29.953465 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260808 03:00:29.953825 20484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
I20260808 03:00:29.955569 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:29.955200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53724)
0808 03:00:29.955359 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:29.955431 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 03:00:29.955780 20555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:46485: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab.
Entry for principal kudu/127.0.205.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab.
I20260808 03:00:29.999042 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.189:46485
--webserver_interface=127.0.205.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab
--principal=kudu/127.0.205.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:43921,127.0.205.189:46485,127.0.205.188:37425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:00:30.005214 20555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } attempt: 1
I20260808 03:00:30.006608 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.006083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53736)
0808 03:00:30.006333 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.006468 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 03:00:30.007732 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.007346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53742)
0808 03:00:30.007506 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.007588 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 03:00:30.007936 20555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:46485: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
I20260808 03:00:30.076277 20555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } attempt: 2
I20260808 03:00:30.077838 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.077122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53750)
0808 03:00:30.077482 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.077631 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260808 03:00:30.079233 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.078693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53754)
0808 03:00:30.078913 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.079013 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 03:00:30.079505 20555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:46485: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
I20260808 03:00:30.170801 20555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } attempt: 3
I20260808 03:00:30.172266 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.171587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53760)
0808 03:00:30.171878 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.172029 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 03:00:30.173571 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.172977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53764)
0808 03:00:30.173211 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.173331 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 03:00:30.173823 20555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:46485: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
I20260808 03:00:30.317126 20555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } attempt: 4
I20260808 03:00:30.318770 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.317979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53776)
0808 03:00:30.318434 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.318564 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205}
I20260808 03:00:30.320046 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.319687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53784)
0808 03:00:30.319828 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.319929 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 03:00:30.320230 20555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:46485: Network error: Client connection negotiation failed: client connection to 127.0.205.189:46485: connect: Connection refused (error 111)
W20260808 03:00:30.376859 20567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:30.377259 20567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:30.377425 20567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:30.385146 20567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:00:30.385349 20567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:30.385476 20567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:30.385586 20567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:00:30.385696 20567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:00:30.396289 20567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:33833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:43921,127.0.205.189:46485,127.0.205.188:37425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab
--principal=kudu/127.0.205.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.189:46485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.205.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:30.397785 20567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:30.399276 20567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:30.411989 20577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:30.412082 20574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:30.412652 20573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:30.413484 20567 server_base.cc:1034] running on GCE node
I20260808 03:00:30.414445 20567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:30.415952 20567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:30.417171 20567 hybrid_clock.cc:648] HybridClock initialized: now 1786158030417119 us; error 45 us; skew 500 ppm
I20260808 03:00:30.417615 20567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:30 dist-test-slave-6ntk krb5kdc[20436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:30.420965 20567 init.cc:377] Logged in from keytab as kudu/127.0.205.189@KRBTEST.COM (short username kudu)
I20260808 03:00:30.422693 20567 webserver.cc:460] Webserver started at http://127.0.205.189:38193/ using document root <none> and password file <none>
I20260808 03:00:30.423442 20567 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:30.423596 20567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:30.423918 20567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:00:30.427254 20567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/data/instance:
uuid: "baa57aa7b9d3471e82f94e1aa27ed505"
format_stamp: "Formatted at 2026-08-08 03:00:30 on dist-test-slave-6ntk"
server_key: "f852f2ca936079d8299b4dfc273ff4df"
server_key_iv: "39c952bf14fee57c1da54a9634c313ec"
server_key_version: "encryptionkey@0"
I20260808 03:00:30.428156 20567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/wal/instance:
uuid: "baa57aa7b9d3471e82f94e1aa27ed505"
format_stamp: "Formatted at 2026-08-08 03:00:30 on dist-test-slave-6ntk"
server_key: "f852f2ca936079d8299b4dfc273ff4df"
server_key_iv: "39c952bf14fee57c1da54a9634c313ec"
server_key_version: "encryptionkey@0"
I20260808 03:00:30.433712 20567 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 03:00:30.437374 20588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:30.438915 20567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:00:30.439123 20567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/wal
uuid: "baa57aa7b9d3471e82f94e1aa27ed505"
format_stamp: "Formatted at 2026-08-08 03:00:30 on dist-test-slave-6ntk"
server_key: "f852f2ca936079d8299b4dfc273ff4df"
server_key_iv: "39c952bf14fee57c1da54a9634c313ec"
server_key_version: "encryptionkey@0"
I20260808 03:00:30.439354 20567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:30.460948 20567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:30.466039 20567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:30.466377 20567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:30.486335 20567 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.189:46485
I20260808 03:00:30.486367 20664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.189:46485 every 8 connection(s)
I20260808 03:00:30.487879 20567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
I20260808 03:00:30.490077 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 20567
I20260808 03:00:30.490361 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-1/wal/instance
I20260808 03:00:30.490868 822 external_mini_cluster.cc:1428] Setting key d278d8e0b94a53f203b167d60d15def5
I20260808 03:00:30.492496 20665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:30.500386 20665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:00:30 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.205.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.188@KRBTEST.COM" created.
I20260808 03:00:30.529857 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.504551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58918 (local address 127.0.205.190:43921)
0808 03:00:30.505228 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:30.505235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:30.506468 (+ 1233us) server_negotiation.cc:407] Connection header received
0808 03:00:30.509997 (+ 3529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:30.510002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:30.510094 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:30.510195 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:30.512757 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.513568 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.516640 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.517212 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.517355 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:30.522559 (+ 5204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:30.522566 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:30.522570 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:30.522616 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:30.526084 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:30.527036 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:30.527045 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:30.527051 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:30.527226 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:30.527689 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:30.527699 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:30.527705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:30.528157 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:30.528336 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:30.529073 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 03:00:30.529356 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":368,"threads_started":1}
I20260808 03:00:30.530647 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.505195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43921 (local address 127.0.0.1:58918)
0808 03:00:30.506241 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.506301 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:30.509546 (+ 3245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:30.510532 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:30.510548 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:30.511550 (+ 1002us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:30.512528 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:30.512562 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.513783 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.513792 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:30.516419 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:30.516444 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.517545 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.517569 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:30.517990 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:30.518877 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:30.518921 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:30.522205 (+ 3284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:30.526300 (+ 4095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:30.526310 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:30.526327 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:30.526841 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:30.527355 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:30.527359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:30.527361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:30.527560 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:30.528290 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:30.528297 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:30.528679 (+ 382us) client_negotiation.cc:769] Sending connection context
0808 03:00:30.528934 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 03:00:30.529360 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":362,"threads_started":1}
I20260808 03:00:30.532605 20665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:00:30 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:00:30.550858 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.533509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53800 (local address 127.0.205.189:46485)
0808 03:00:30.534247 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:30.534258 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:30.534293 (+ 35us) server_negotiation.cc:407] Connection header received
0808 03:00:30.534366 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:30.534377 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:30.534625 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:30.534799 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:30.535542 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.537278 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.539171 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.539895 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.540047 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:30.544256 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:30.544273 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:30.544291 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:30.544340 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:30.547502 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:30.548476 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:30.548483 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:30.548485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:30.548615 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:30.549107 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:30.549111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:30.549113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:30.549496 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:30.549673 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:30.550214 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 03:00:30.550542 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":530,"thread_start_us":361,"threads_started":1}
I20260808 03:00:30.550956 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.533483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53800)
0808 03:00:30.533704 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.533730 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:30.533905 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:30.534792 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:30.534795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:30.535185 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:30.535347 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:30.535356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.537440 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.537448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:30.539016 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:30.539034 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.540088 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.540094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:30.540377 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:30.541151 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:30.541169 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:30.543947 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:30.547682 (+ 3735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:30.547691 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:30.547698 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:30.548316 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:30.548733 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:30.548741 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:30.548748 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:30.548980 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:30.549638 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:30.549646 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:30.549843 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 03:00:30.550042 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 03:00:30.550435 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 03:00:30.553732 20665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:00:30.555279 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.554619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49000)
0808 03:00:30.554848 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.555032 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260808 03:00:30.555631 20593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
I20260808 03:00:30.557432 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.557064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49008)
0808 03:00:30.557223 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.557318 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 03:00:30.557696 20665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:37425: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab.
Entry for principal kudu/127.0.205.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab.
I20260808 03:00:30.565552 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.188:37425
--webserver_interface=127.0.205.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab
--principal=kudu/127.0.205.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:43921,127.0.205.189:46485,127.0.205.188:37425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:00:30.607072 20665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } attempt: 1
I20260808 03:00:30.608642 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.607882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49010)
0808 03:00:30.608246 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.608405 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 03:00:30.609746 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.609398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49026)
0808 03:00:30.609555 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.609632 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 03:00:30.610021 20665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:37425: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
I20260808 03:00:30.619465 20555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } attempt: 5
Aug 08 03:00:30 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:00:30.635568 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.620321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.0.205.189:46485)
0808 03:00:30.620498 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:30.620502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:30.620585 (+ 83us) server_negotiation.cc:407] Connection header received
0808 03:00:30.620896 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:30.620900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:30.620966 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:30.621045 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:30.622218 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.622953 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.624849 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.625437 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.625537 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:30.629964 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:30.629969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:30.629973 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:30.630008 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:30.632723 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:30.633494 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:30.633501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:30.633504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:30.633638 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:30.634110 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:30.634119 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:30.634121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:30.634416 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:30.634585 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:30.635003 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 03:00:30.635235 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:00:30.635589 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.620178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53802)
0808 03:00:30.620459 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.620488 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:30.620672 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:30.621132 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:30.621140 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:30.621738 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:30.622054 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:30.622072 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.623098 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.623104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:30.624664 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:30.624695 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:30.625605 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:30.625611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:30.625853 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:30.626664 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:30.626708 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:30.629663 (+ 2955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:30.632935 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:30.632941 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:30.632946 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:30.633358 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:30.633760 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:30.633766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:30.633770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:30.633985 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:30.634547 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:30.634553 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:30.634724 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 03:00:30.634873 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 03:00:30.635221 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 03:00:30.636732 20555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:00:30.638015 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.637447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49040)
0808 03:00:30.637658 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.637778 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 03:00:30.639061 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.638645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49054)
0808 03:00:30.638808 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.638922 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 03:00:30.639324 20555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:37425: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
I20260808 03:00:30.678242 20665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } attempt: 2
I20260808 03:00:30.679555 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.679053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49056)
0808 03:00:30.679273 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.679382 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 03:00:30.680632 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.680265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49058)
0808 03:00:30.680421 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.680508 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 03:00:30.680876 20665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:37425: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
I20260808 03:00:30.690584 20555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } attempt: 1
I20260808 03:00:30.691887 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.691388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49064)
0808 03:00:30.691590 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.691695 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 03:00:30.693046 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.692668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49074)
0808 03:00:30.692860 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.692935 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 03:00:30.693279 20555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:37425: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
I20260808 03:00:30.761505 20555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } attempt: 2
I20260808 03:00:30.762804 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.762284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49090)
0808 03:00:30.762535 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.762633 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 03:00:30.763901 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.763544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49102)
0808 03:00:30.763707 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.763777 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 03:00:30.764108 20555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:37425: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
I20260808 03:00:30.772079 20665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } attempt: 3
I20260808 03:00:30.773227 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.772710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49116)
0808 03:00:30.772967 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.773073 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 03:00:30.774363 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.773931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49122)
0808 03:00:30.774159 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.774242 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260808 03:00:30.774557 20665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:37425: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
I20260808 03:00:30.870321 20555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } attempt: 3
I20260808 03:00:30.871590 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.871097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49138)
0808 03:00:30.871295 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.871421 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 03:00:30.872757 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.872322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49154)
0808 03:00:30.872562 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.872642 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260808 03:00:30.872962 20555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:37425: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
I20260808 03:00:30.917784 20665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } attempt: 4
I20260808 03:00:30.919090 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.918568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49160)
0808 03:00:30.918821 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.918924 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 03:00:30.920156 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:30.919801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49168)
0808 03:00:30.919963 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:30.920046 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 03:00:30.920394 20665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:37425: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
W20260808 03:00:30.937142 20676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:30.937533 20676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:30.937696 20676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:30.946041 20676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:00:30.946197 20676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:30.946298 20676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:30.946408 20676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:00:30.946484 20676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:00:30.956701 20676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:33833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:43921,127.0.205.189:46485,127.0.205.188:37425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab
--principal=kudu/127.0.205.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.188:37425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.205.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:30.958243 20676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:30.959571 20676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:30.972695 20688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:30.973002 20685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:30.972716 20684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:30.978614 20676 server_base.cc:1034] running on GCE node
I20260808 03:00:30.979609 20676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:30.981238 20676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:30.982487 20676 hybrid_clock.cc:648] HybridClock initialized: now 1786158030982409 us; error 70 us; skew 500 ppm
I20260808 03:00:30.982865 20676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:30 dist-test-slave-6ntk krb5kdc[20436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:30.986514 20676 init.cc:377] Logged in from keytab as kudu/127.0.205.188@KRBTEST.COM (short username kudu)
I20260808 03:00:30.988334 20676 webserver.cc:460] Webserver started at http://127.0.205.188:45509/ using document root <none> and password file <none>
I20260808 03:00:30.989138 20676 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:30.989295 20676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:30.989642 20676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:00:30.993014 20676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/data/instance:
uuid: "8692effaca5042a8a63b5d3d0f15da22"
format_stamp: "Formatted at 2026-08-08 03:00:30 on dist-test-slave-6ntk"
server_key: "d2b2abd22a8a4d92658ad994a52fabff"
server_key_iv: "2d99833378a3f7174cd7276c769f7149"
server_key_version: "encryptionkey@0"
I20260808 03:00:30.994004 20676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/wal/instance:
uuid: "8692effaca5042a8a63b5d3d0f15da22"
format_stamp: "Formatted at 2026-08-08 03:00:30 on dist-test-slave-6ntk"
server_key: "d2b2abd22a8a4d92658ad994a52fabff"
server_key_iv: "2d99833378a3f7174cd7276c769f7149"
server_key_version: "encryptionkey@0"
I20260808 03:00:30.999786 20676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:00:31.003454 20700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:31.004907 20676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:00:31.005116 20676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/wal
uuid: "8692effaca5042a8a63b5d3d0f15da22"
format_stamp: "Formatted at 2026-08-08 03:00:30 on dist-test-slave-6ntk"
server_key: "d2b2abd22a8a4d92658ad994a52fabff"
server_key_iv: "2d99833378a3f7174cd7276c769f7149"
server_key_version: "encryptionkey@0"
I20260808 03:00:31.005403 20676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:31.026718 20676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:31.031405 20676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:31.031704 20676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:31.050172 20555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } attempt: 4
I20260808 03:00:31.051508 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.050918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49182)
0808 03:00:31.051132 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:31.051286 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:37425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 03:00:31.052315 20676 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.188:37425
I20260808 03:00:31.052327 20772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.188:37425 every 8 connection(s)
I20260808 03:00:31.054005 20676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
I20260808 03:00:31.055531 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 20676
I20260808 03:00:31.055747 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786157811617362-822-0/minicluster-data/master-2/wal/instance
I20260808 03:00:31.056317 822 external_mini_cluster.cc:1428] Setting key f89881f800a067b84fa0f3be8f0581d5
I20260808 03:00:31.059258 20774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 03:00:31 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
Aug 08 03:00:31 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
I20260808 03:00:31.068958 20774 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:00:31.077558 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.052265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49192)
0808 03:00:31.052450 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:31.052468 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:31.052602 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:31.058323 (+ 5721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:31.058329 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:31.058924 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:31.059156 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:31.059173 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.062181 (+ 3008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.062188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:31.063635 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:31.063653 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.064756 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.064764 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:31.064955 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.065480 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:31.065509 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:31.068568 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:31.073813 (+ 5245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:31.073823 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:31.073830 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:31.074492 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:31.075081 (+ 589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:31.075091 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:31.075094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:31.075330 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:31.076409 (+ 1079us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:31.076418 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:31.076632 (+ 214us) client_negotiation.cc:769] Sending connection context
0808 03:00:31.076818 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 03:00:31.077148 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 03:00:31.079493 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.053657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49192 (local address 127.0.205.188:37425)
0808 03:00:31.054658 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.054680 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.054750 (+ 70us) server_negotiation.cc:407] Connection header received
0808 03:00:31.057229 (+ 2479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.057261 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.057799 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.059335 (+ 1536us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.059814 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.061982 (+ 2168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.063848 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.064501 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.064690 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.068871 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.068879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.068897 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.068944 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.073602 (+ 4658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.074655 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.074663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.074670 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.074927 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.075456 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.075463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.075467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.076236 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.076444 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.077092 (+ 648us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.077543 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":688,"spinlock_wait_cycles":18816,"thread_start_us":460,"threads_started":1}
I20260808 03:00:31.079523 20779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.057716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.0.205.188:37425)
0808 03:00:31.058781 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.058785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.058805 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:00:31.058844 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.058847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.058895 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.059119 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.060834 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.061979 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.063380 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.064107 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.064338 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.069784 (+ 5446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.069793 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.069796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.069830 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.074781 (+ 4951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.075768 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.075774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.075777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.075945 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.076400 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.076405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.076412 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.076765 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.077071 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.078058 (+ 987us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.078303 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":950,"thread_start_us":766,"threads_started":1}
Aug 08 03:00:31 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:31.087878 20555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03: Bootstrap starting.
I20260808 03:00:31.092379 20555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:31.092712 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.072665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43921 (local address 127.0.0.1:58934)
0808 03:00:31.074534 (+ 1869us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:31.074583 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:31.074794 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:31.075270 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:31.075279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:31.076143 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:31.076582 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:31.076597 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.078386 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.078392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:31.079755 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:31.079776 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.080699 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.080706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:31.080911 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.081743 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:31.081773 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:31.084894 (+ 3121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:31.089247 (+ 4353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:31.089266 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:31.089274 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:31.089744 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:31.090218 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:31.090223 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:31.090227 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:31.090450 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:31.091319 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:31.091330 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:31.091551 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 03:00:31.091830 (+ 279us) client_negotiation.cc:240] Negotiation successful
0808 03:00:31.092248 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1587,"thread_start_us":1395,"threads_started":1}
I20260808 03:00:31.093837 20780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.072569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58934 (local address 127.0.205.190:43921)
0808 03:00:31.073242 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.073251 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.074911 (+ 1660us) server_negotiation.cc:407] Connection header received
0808 03:00:31.074975 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.074980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.075074 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.075202 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.076751 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.078192 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.079989 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.080491 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.080592 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.085152 (+ 4560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.085165 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.085171 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.085211 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.089089 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.089933 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.089939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.089944 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.090087 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.090574 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.090579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.090583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.091065 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.091214 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.093090 (+ 1876us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.093336 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":295,"threads_started":1}
I20260808 03:00:31.093948 20555 log.cc:824] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03: Log is configured to *not* fsync() on all Append() calls
Aug 08 03:00:31 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:31.095268 20774 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:00:31 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:00:31.104313 20555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03: No bootstrap required, opened a new log
I20260808 03:00:31.105628 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.083901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58936 (local address 127.0.205.190:43921)
0808 03:00:31.084619 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.084625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.084656 (+ 31us) server_negotiation.cc:407] Connection header received
0808 03:00:31.084724 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.084729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.084835 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.084965 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.086899 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.088434 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.090407 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.090965 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.091107 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.097210 (+ 6103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.097220 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.097228 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.097283 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.101917 (+ 4634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.102937 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.102944 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.102949 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.103126 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.103633 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.103639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.103643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.103972 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.104286 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.105147 (+ 861us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.105333 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":322,"threads_started":1}
I20260808 03:00:31.112290 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.096139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:46485 (local address 127.0.0.1:53806)
0808 03:00:31.096507 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:31.096522 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:31.096659 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:31.097017 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:31.097022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:31.097435 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:31.097576 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:31.097584 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.098873 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.098878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:31.099754 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:31.099797 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.100774 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.100779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:31.100912 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.101588 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:31.101614 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:31.104013 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:31.108772 (+ 4759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:31.108778 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:31.108782 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:31.109416 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:31.110167 (+ 751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:31.110173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:31.110176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:31.110371 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:31.111189 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:31.111195 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:31.111385 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 03:00:31.111559 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 03:00:31.111883 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252}
I20260808 03:00:31.112784 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.096187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.0.205.189:46485)
0808 03:00:31.096374 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.096381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.096601 (+ 220us) server_negotiation.cc:407] Connection header received
0808 03:00:31.096802 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.096808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.096885 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.096991 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.097731 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.098686 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.099970 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.100481 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.100561 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.104263 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.104269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.104274 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.104324 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.108574 (+ 4250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.109712 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.109721 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.109727 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.110016 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.110550 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.110558 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.110563 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.111028 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.111227 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.111707 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.112019 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:00:31.114068 20774 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:00:31.115527 20555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } }
I20260808 03:00:31.115988 20555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:31.116127 20555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19897f17af3a47b985941d42ee483f03, State: Initialized, Role: FOLLOWER
I20260808 03:00:31.116901 20555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } }
I20260808 03:00:31.118479 20788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } } }
I20260808 03:00:31.118901 20788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:00:31.120031 20555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 03:00:31 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
Aug 08 03:00:31 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:00:31.131999 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.114971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49214 (local address 127.0.205.188:37425)
0808 03:00:31.115118 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.115125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.115184 (+ 59us) server_negotiation.cc:407] Connection header received
0808 03:00:31.115359 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.115364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.115431 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.115519 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.116317 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.117386 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.119113 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.119715 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.119845 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.124939 (+ 5094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.124949 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.124957 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.125025 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.128726 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.129558 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.129563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.129566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.129759 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.130257 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.130263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.130269 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.130719 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.130844 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.131383 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.131612 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 03:00:31.132671 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.107791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.0.205.189:46485)
0808 03:00:31.114147 (+ 6356us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.114152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.114175 (+ 23us) server_negotiation.cc:407] Connection header received
0808 03:00:31.114226 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.114230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.114320 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.114422 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.116328 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.117244 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.119307 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.119718 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.119863 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.125108 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.125113 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.125116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.125147 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.129036 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.130030 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.130037 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.130042 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.130220 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.130755 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.130759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.130763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.131106 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.131379 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.131958 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.132304 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6235,"thread_start_us":7521,"threads_started":1}
I20260808 03:00:31.132879 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.114870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49214)
0808 03:00:31.115116 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:31.115135 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:31.115240 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:31.115534 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:31.115538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:31.115975 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:31.116167 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:31.116182 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.117544 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.117550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:31.118930 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:31.118946 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.120048 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.120057 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:31.120274 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.121010 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:31.121035 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:31.124685 (+ 3650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:31.128894 (+ 4209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:31.128899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:31.128904 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:31.129443 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:31.129970 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:31.129977 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:31.129983 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:31.130146 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:31.130947 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:31.130953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:31.131116 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 03:00:31.132211 (+ 1095us) client_negotiation.cc:240] Negotiation successful
0808 03:00:31.132534 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260808 03:00:31.138152 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:00:31.139029 20774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22: Bootstrap starting.
W20260808 03:00:31.140409 20490 tablet.cc:2369] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:00:31.143378 20774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:31.144789 20774 log.cc:824] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:31.135120 20458 master_runner.cc:186] Error getting master registration for 127.0.205.189:46485: OK, instance_id { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" instance_seqno: 1786158030480418 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:00:31.146370 20458 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:00:31.149535 20774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22: No bootstrap required, opened a new log
W20260808 03:00:31.152515 20803 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:00:31.152714 20803 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:00:31.157651 20774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } }
I20260808 03:00:31.158098 20774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:31.158243 20774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8692effaca5042a8a63b5d3d0f15da22, State: Initialized, Role: FOLLOWER
I20260808 03:00:31.159102 20774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } }
I20260808 03:00:31.160104 20810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } } }
I20260808 03:00:31.160594 20810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:00:31.161729 20774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:00:31.177790 20788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:00:31.178194 20788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } }
I20260808 03:00:31.180089 20788 leader_election.cc:288] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baa57aa7b9d3471e82f94e1aa27ed505 (127.0.205.189:46485), 8692effaca5042a8a63b5d3d0f15da22 (127.0.205.188:37425)
I20260808 03:00:31.181562 20738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19897f17af3a47b985941d42ee483f03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8692effaca5042a8a63b5d3d0f15da22" is_pre_election: true
I20260808 03:00:31.182299 20738 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19897f17af3a47b985941d42ee483f03 in term 0.
I20260808 03:00:31.183151 20487 leader_election.cc:302] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19897f17af3a47b985941d42ee483f03, 8692effaca5042a8a63b5d3d0f15da22; no voters:
I20260808 03:00:31.183694 20788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:00:31.183867 20788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:00:31.183983 20788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:31.187747 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.161016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58940 (local address 127.0.205.190:43921)
0808 03:00:31.161228 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.161235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.161846 (+ 611us) server_negotiation.cc:407] Connection header received
0808 03:00:31.164110 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.164116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.164214 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.164302 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.167123 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.168228 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.170755 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.171543 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.171647 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.175838 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.175846 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.175850 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.175906 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.181149 (+ 5243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.182405 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.182414 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.182420 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.183654 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.184319 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.184328 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.184334 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.184907 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.186587 (+ 1680us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.187064 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.187354 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:00:31.190615 20788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } }
I20260808 03:00:31.195788 20738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19897f17af3a47b985941d42ee483f03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8692effaca5042a8a63b5d3d0f15da22"
I20260808 03:00:31.196069 20738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:31.201690 20738 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19897f17af3a47b985941d42ee483f03 in term 1.
I20260808 03:00:31.202533 20487 leader_election.cc:302] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19897f17af3a47b985941d42ee483f03, 8692effaca5042a8a63b5d3d0f15da22; no voters:
I20260808 03:00:31.209913 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.162213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53826 (local address 127.0.205.189:46485)
0808 03:00:31.162860 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.162868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.162972 (+ 104us) server_negotiation.cc:407] Connection header received
0808 03:00:31.163254 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.163260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.163347 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.163436 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.166087 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.167207 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.171161 (+ 3954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.172130 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.172270 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.193636 (+ 21366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.193648 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.193656 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.193717 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.198649 (+ 4932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.199888 (+ 1239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.199898 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.199904 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.200161 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.200754 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.200775 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.200780 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.201262 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.201566 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.208863 (+ 7297us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.209305 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514}
I20260808 03:00:31.209618 20788 leader_election.cc:288] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [CANDIDATE]: Term 1 election: Requested vote from peers baa57aa7b9d3471e82f94e1aa27ed505 (127.0.205.189:46485), 8692effaca5042a8a63b5d3d0f15da22 (127.0.205.188:37425)
I20260808 03:00:31.211288 20788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:00:31.212281 20788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [term 1 LEADER]: Becoming Leader. State: Replica: 19897f17af3a47b985941d42ee483f03, State: Running, Role: LEADER
I20260808 03:00:31.215641 20788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } }
I20260808 03:00:31.216364 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.162808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.0.205.188:37425)
0808 03:00:31.162992 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.163000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.164035 (+ 1035us) server_negotiation.cc:407] Connection header received
0808 03:00:31.164278 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.164284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.164356 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.164423 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.167857 (+ 3434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.168890 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.187555 (+ 18665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.188533 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.188670 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.202383 (+ 13713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.202394 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.202402 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.202461 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.207370 (+ 4909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.212794 (+ 5424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.212804 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.212810 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.213032 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.213670 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.213679 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.213685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.214265 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.214706 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.215472 (+ 766us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.215820 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 03:00:31.219589 20665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } attempt: 5
I20260808 03:00:31.232167 20824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19897f17af3a47b985941d42ee483f03. Latest consensus state: current_term: 1 leader_uuid: "19897f17af3a47b985941d42ee483f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } } }
I20260808 03:00:31.232651 20824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [sys.catalog]: This master's current role is: LEADER
Aug 08 03:00:31 dist-test-slave-6ntk krb5kdc[20436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
I20260808 03:00:31.238847 20828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:00:31.246968 20828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:00:31.249130 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.220289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:37425 (local address 127.0.0.1:49230)
0808 03:00:31.220745 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:31.220785 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:31.220943 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:31.221914 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:31.221924 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:31.222614 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:31.222959 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:31.222977 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.228215 (+ 5238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.228225 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:31.230286 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:31.230308 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.231730 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.231739 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:31.232063 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.232981 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:31.233015 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:31.236908 (+ 3893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:31.243996 (+ 7088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:31.244005 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:31.244012 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:31.244813 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:31.245424 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:31.245432 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:31.245439 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:31.245661 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:31.246635 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:31.246643 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:31.246882 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 03:00:31.247053 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 03:00:31.248699 (+ 1646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291}
I20260808 03:00:31.250239 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:31.221216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.0.205.188:37425)
0808 03:00:31.221440 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:31.221448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:31.221469 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:00:31.221532 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:31.221540 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:31.221657 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:31.223159 (+ 1502us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:31.223346 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.228013 (+ 4667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.230548 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:31.231516 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:31.237153 (+ 5637us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:31.237455 (+ 302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:31.237464 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:31.237472 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:31.237512 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:31.243800 (+ 6288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.245036 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.245044 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.245051 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.245263 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:31.245850 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:31.245857 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:31.245863 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:31.246456 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:31.249406 (+ 2950us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:31.249566 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 03:00:31.249827 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:00:31.262797 20665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505: Bootstrap starting.
I20260808 03:00:31.268657 20665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:31.275146 20665 log.cc:824] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505: Log is configured to *not* fsync() on all Append() calls
W20260808 03:00:31.283840 20708 tablet.cc:2369] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:00:31.288120 20665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505: No bootstrap required, opened a new log
I20260808 03:00:31.290934 20738 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [term 1 FOLLOWER]: Refusing update from remote peer 19897f17af3a47b985941d42ee483f03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:00:31.292279 20824 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:00:31.296056 20841 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 03:00:31.309470 20665 raft_consensus.cc:348] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } }
I20260808 03:00:31.310364 20665 raft_consensus.cc:374] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:31.310590 20665 raft_consensus.cc:729] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa57aa7b9d3471e82f94e1aa27ed505, State: Initialized, Role: FOLLOWER
I20260808 03:00:31.311836 20665 consensus_queue.cc:260] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } }
I20260808 03:00:31.303936 20676 master_runner.cc:186] Error getting master registration for 127.0.205.189:46485: OK, instance_id { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" instance_seqno: 1786158030480418 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:00:31.318259 20676 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:00:31.319890 20810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19897f17af3a47b985941d42ee483f03. Latest consensus state: current_term: 1 leader_uuid: "19897f17af3a47b985941d42ee483f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } } }
I20260808 03:00:31.320194 20810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:00:31.322229 20624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19897f17af3a47b985941d42ee483f03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" is_pre_election: true
I20260808 03:00:31.322556 20621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:31.323140 20622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19897f17af3a47b985941d42ee483f03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa57aa7b9d3471e82f94e1aa27ed505"
I20260808 03:00:31.323493 20834 mvcc.cc:205] Tried to move back new op lower bound from 7316103296150446080 to 7316103295872524288. Current Snapshot: MvccSnapshot[applied={T|T < 7316103296150446080}]
I20260808 03:00:31.323530 20665 sys_catalog.cc:564] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:00:31.326967 20621 raft_consensus.cc:1261] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [term 1 FOLLOWER]: Refusing update from remote peer 19897f17af3a47b985941d42ee483f03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:00:31.328099 20824 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:00:31.331135 20810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "19897f17af3a47b985941d42ee483f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } } }
I20260808 03:00:31.331403 20810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:00:31.339349 20824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "19897f17af3a47b985941d42ee483f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } } }
I20260808 03:00:31.339732 20824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:31.341792 20828 catalog_manager.cc:1347] Generated new cluster ID: 816b658c5fa54df1ac1e6db4c0ee7a16
I20260808 03:00:31.341944 20828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:00:31.365110 20843 mvcc.cc:205] Tried to move back new op lower bound from 7316103296150446080 to 7316103295872524288. Current Snapshot: MvccSnapshot[applied={T|T < 7316103296150446080}]
I20260808 03:00:31.376099 20844 sys_catalog.cc:455] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19897f17af3a47b985941d42ee483f03. Latest consensus state: current_term: 1 leader_uuid: "19897f17af3a47b985941d42ee483f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } } }
I20260808 03:00:31.376557 20844 sys_catalog.cc:458] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:00:31.376830 20848 sys_catalog.cc:455] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19897f17af3a47b985941d42ee483f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } } }
I20260808 03:00:31.377264 20848 sys_catalog.cc:458] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:00:31.376849 20856 sys_catalog.cc:455] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "19897f17af3a47b985941d42ee483f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } } }
I20260808 03:00:31.377907 20856 sys_catalog.cc:458] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:00:31.387632 20874 catalog_manager.cc:1259] Loaded cluster ID: 816b658c5fa54df1ac1e6db4c0ee7a16
I20260808 03:00:31.387831 20874 catalog_manager.cc:1552] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505: loading cluster ID for follower catalog manager: success
W20260808 03:00:31.389283 20874 catalog_manager.cc:1576] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 03:00:31.389595 20874 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:00:31.410923 20824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "19897f17af3a47b985941d42ee483f03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19897f17af3a47b985941d42ee483f03" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43921 } } peers { permanent_uuid: "baa57aa7b9d3471e82f94e1aa27ed505" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 46485 } } peers { permanent_uuid: "8692effaca5042a8a63b5d3d0f15da22" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 37425 } } }
I20260808 03:00:31.411273 20824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:31.414876 20828 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:00:31.416889 20828 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:00:31.430310 20828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19897f17af3a47b985941d42ee483f03: Generated new TSK 0
I20260808 03:00:31.431604 20828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:00:32.300840 20841 catalog_manager.cc:1259] Loaded cluster ID: 816b658c5fa54df1ac1e6db4c0ee7a16
I20260808 03:00:32.300973 20841 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22: loading cluster ID for follower catalog manager: success
I20260808 03:00:32.303625 20841 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22: acquiring CA information for follower catalog manager: success
I20260808 03:00:32.305588 20841 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8692effaca5042a8a63b5d3d0f15da22: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 03:00:32.393035 20874 catalog_manager.cc:1574] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505: acquiring CA information for follower catalog manager: success
I20260808 03:00:32.395599 20874 catalog_manager.cc:1602] T 00000000000000000000000000000000 P baa57aa7b9d3471e82f94e1aa27ed505: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 03:00:32.533846 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 20458
I20260808 03:00:32.535498 20547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:32.681937 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 20458
W20260808 03:00:32.714285 20706 connection.cc:535] server connection from 127.0.0.1:49192 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 03:00:32.714948 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 20567
I20260808 03:00:32.716147 20653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:32.826699 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 20567
I20260808 03:00:32.855762 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 20676
I20260808 03:00:32.856953 20763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:32.968426 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 20676
2026-08-08T03:00:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5261 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:00:33 dist-test-slave-6ntk krb5kdc[20891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:00:33 dist-test-slave-6ntk krb5kdc[20891](info): set up 2 sockets
Aug 08 03:00:33 dist-test-slave-6ntk krb5kdc[20891](info): commencing operation
krb5kdc: starting...
W20260808 03:00:34.324268 822 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:00:34 dist-test-slave-6ntk krb5kdc[20891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:00:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:00:34.630573 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:40967
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:32987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:40967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:35.006963 20915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:35.007371 20915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:35.007542 20915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:35.015075 20915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:00:35.015201 20915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:35.015304 20915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:35.015419 20915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:00:35.015506 20915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:00:35.025769 20915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:32987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:40967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:40967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:35.027251 20915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:35.028626 20915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:35.041471 20923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:35.041502 20926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:35.041544 20924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:35.042215 20915 server_base.cc:1034] running on GCE node
I20260808 03:00:35.043174 20915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:35.044819 20915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:35.046060 20915 hybrid_clock.cc:648] HybridClock initialized: now 1786158035046010 us; error 61 us; skew 500 ppm
I20260808 03:00:35.046453 20915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:35 dist-test-slave-6ntk krb5kdc[20891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158035, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:35.055572 20915 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:00:35.057416 20915 webserver.cc:460] Webserver started at http://127.0.205.190:37435/ using document root <none> and password file <none>
I20260808 03:00:35.058307 20915 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:35.058478 20915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:35.058833 20915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:00:35.062247 20915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "a21dfe631dd746018d7c9a5928d1c231"
format_stamp: "Formatted at 2026-08-08 03:00:35 on dist-test-slave-6ntk"
server_key: "48628bb36bf8d984c3d513ac1ef8853c"
server_key_iv: "395283cdc895da79bd4ead16ec62b220"
server_key_version: "encryptionkey@0"
I20260808 03:00:35.063107 20915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "a21dfe631dd746018d7c9a5928d1c231"
format_stamp: "Formatted at 2026-08-08 03:00:35 on dist-test-slave-6ntk"
server_key: "48628bb36bf8d984c3d513ac1ef8853c"
server_key_iv: "395283cdc895da79bd4ead16ec62b220"
server_key_version: "encryptionkey@0"
I20260808 03:00:35.068946 20915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 03:00:35.072942 20935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:35.074450 20915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:00:35.074658 20915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "a21dfe631dd746018d7c9a5928d1c231"
format_stamp: "Formatted at 2026-08-08 03:00:35 on dist-test-slave-6ntk"
server_key: "48628bb36bf8d984c3d513ac1ef8853c"
server_key_iv: "395283cdc895da79bd4ead16ec62b220"
server_key_version: "encryptionkey@0"
I20260808 03:00:35.074887 20915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:35.090574 20915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:35.095929 20915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:35.096261 20915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:35.117005 20915 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:40967
I20260808 03:00:35.117045 21017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:40967 every 8 connection(s)
I20260808 03:00:35.118709 20915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:00:35.121122 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 20915
I20260808 03:00:35.121419 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:00:35.122328 822 external_mini_cluster.cc:1428] Setting key 6248a19941d2f3aee9ff398634d2af16
I20260808 03:00:35.123957 21018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 03:00:35 dist-test-slave-6ntk krb5kdc[20891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:35.139739 21018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231: Bootstrap starting.
I20260808 03:00:35.143352 21018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:35.144766 21018 log.cc:824] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:35.148255 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:35.124401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55054 (local address 127.0.205.190:40967)
0808 03:00:35.125242 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:35.125255 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:35.125299 (+ 44us) server_negotiation.cc:407] Connection header received
0808 03:00:35.127042 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:35.127066 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:35.127507 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:35.127862 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:35.129983 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:35.131424 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:35.133393 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:35.134235 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:35.134384 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:35.140089 (+ 5705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:35.140097 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:35.140115 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:35.140163 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:35.143503 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:35.144532 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:35.144539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:35.144545 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:35.144741 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:35.145205 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:35.145208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:35.145210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:35.146065 (+ 855us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:35.146437 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:35.146810 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 03:00:35.147183 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":380,"threads_started":1}
I20260808 03:00:35.148406 21018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231: No bootstrap required, opened a new log
I20260808 03:00:35.152395 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:00:35.158002 21018 raft_consensus.cc:348] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21dfe631dd746018d7c9a5928d1c231" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40967 } }
I20260808 03:00:35.158483 21018 raft_consensus.cc:374] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:35.158609 21018 raft_consensus.cc:729] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a21dfe631dd746018d7c9a5928d1c231, State: Initialized, Role: FOLLOWER
I20260808 03:00:35.159497 21018 consensus_queue.cc:260] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21dfe631dd746018d7c9a5928d1c231" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40967 } }
I20260808 03:00:35.159725 21018 raft_consensus.cc:388] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:00:35.159829 21018 raft_consensus.cc:482] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:00:35.159976 21018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:35.162237 21018 raft_consensus.cc:504] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21dfe631dd746018d7c9a5928d1c231" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40967 } }
I20260808 03:00:35.162719 21018 leader_election.cc:302] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a21dfe631dd746018d7c9a5928d1c231; no voters:
I20260808 03:00:35.163295 21018 leader_election.cc:288] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:00:35.163461 21027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:00:35.164189 21027 raft_consensus.cc:686] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [term 1 LEADER]: Becoming Leader. State: Replica: a21dfe631dd746018d7c9a5928d1c231, State: Running, Role: LEADER
I20260808 03:00:35.164768 21027 consensus_queue.cc:237] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21dfe631dd746018d7c9a5928d1c231" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40967 } }
I20260808 03:00:35.165587 21018 sys_catalog.cc:564] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:00:35.168042 21030 sys_catalog.cc:455] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a21dfe631dd746018d7c9a5928d1c231. Latest consensus state: current_term: 1 leader_uuid: "a21dfe631dd746018d7c9a5928d1c231" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21dfe631dd746018d7c9a5928d1c231" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40967 } } }
I20260808 03:00:35.168551 21030 sys_catalog.cc:458] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:35.168470 21029 sys_catalog.cc:455] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a21dfe631dd746018d7c9a5928d1c231" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a21dfe631dd746018d7c9a5928d1c231" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40967 } } }
I20260808 03:00:35.169041 21029 sys_catalog.cc:458] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:35.170639 21034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:00:35.176337 21034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:00:35.185974 21034 catalog_manager.cc:1347] Generated new cluster ID: 2eb8f97a11ad4aee98afd723a3f869ee
I20260808 03:00:35.186126 21034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:00:35.230638 21034 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:00:35.232165 21034 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:00:35.241912 21034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a21dfe631dd746018d7c9a5928d1c231: Generated new TSK 0
I20260808 03:00:35.242895 21034 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:00:35.267088 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 20915
I20260808 03:00:35.268579 21011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:35.366132 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 20915
2026-08-08T03:00:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2389 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:00:35 dist-test-slave-6ntk krb5kdc[21062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:00:35 dist-test-slave-6ntk krb5kdc[21062](info): set up 2 sockets
Aug 08 03:00:35 dist-test-slave-6ntk krb5kdc[21062](info): commencing operation
krb5kdc: starting...
W20260808 03:00:36.698766 822 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:00:36 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:00:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:00:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:00:37.010105 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:35547
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:35547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:37.374436 21083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:37.374756 21083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:37.374848 21083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:37.382354 21083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:00:37.382478 21083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:37.382527 21083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:37.382587 21083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:00:37.382633 21083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:00:37.392902 21083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:35547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:35547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:37.394313 21083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:37.395653 21083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:37.408453 21092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:37.408419 21096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:37.408545 21094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:37.410071 21083 server_base.cc:1034] running on GCE node
I20260808 03:00:37.410992 21083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:37.412689 21083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:37.414006 21083 hybrid_clock.cc:648] HybridClock initialized: now 1786158037413941 us; error 42 us; skew 500 ppm
I20260808 03:00:37.414512 21083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:37 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158037, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:37.418103 21083 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:00:37.419972 21083 webserver.cc:460] Webserver started at http://127.0.205.190:41229/ using document root <none> and password file <none>
I20260808 03:00:37.420814 21083 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:37.420985 21083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:37.421346 21083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:00:37.424903 21083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "fa049867d0664e22838a4d12f7e50dc3"
format_stamp: "Formatted at 2026-08-08 03:00:37 on dist-test-slave-6ntk"
server_key: "71bb3a64ef4468bb335de202596cf082"
server_key_iv: "bc7ca34db324690da1ab79a970908704"
server_key_version: "encryptionkey@0"
I20260808 03:00:37.425799 21083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "fa049867d0664e22838a4d12f7e50dc3"
format_stamp: "Formatted at 2026-08-08 03:00:37 on dist-test-slave-6ntk"
server_key: "71bb3a64ef4468bb335de202596cf082"
server_key_iv: "bc7ca34db324690da1ab79a970908704"
server_key_version: "encryptionkey@0"
I20260808 03:00:37.432120 21083 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 03:00:37.436033 21106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:37.437570 21083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:00:37.437798 21083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "fa049867d0664e22838a4d12f7e50dc3"
format_stamp: "Formatted at 2026-08-08 03:00:37 on dist-test-slave-6ntk"
server_key: "71bb3a64ef4468bb335de202596cf082"
server_key_iv: "bc7ca34db324690da1ab79a970908704"
server_key_version: "encryptionkey@0"
I20260808 03:00:37.438140 21083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:37.454761 21083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:37.459430 21083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:37.459779 21083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:37.480157 21083 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:35547
I20260808 03:00:37.480189 21172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:35547 every 8 connection(s)
I20260808 03:00:37.481745 21083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:00:37.485970 21175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:37.491600 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 21083
I20260808 03:00:37.491856 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:00:37.492697 822 external_mini_cluster.cc:1428] Setting key 5b91104ec56e42911977c8287346daa8
I20260808 03:00:37.499217 21175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Bootstrap starting.
I20260808 03:00:37.502972 21175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:37.504150 21175 log.cc:824] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:37.507416 21175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: No bootstrap required, opened a new log
Aug 08 03:00:37 dist-test-slave-6ntk krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:37.516436 21175 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:37.516940 21175 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:37.517056 21175 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa049867d0664e22838a4d12f7e50dc3, State: Initialized, Role: FOLLOWER
I20260808 03:00:37.517712 21175 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:37.517988 21175 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:00:37.518087 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:37.495193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34986 (local address 127.0.205.190:35547)
0808 03:00:37.496122 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:37.496135 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:37.496184 (+ 49us) server_negotiation.cc:407] Connection header received
0808 03:00:37.498130 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:37.498159 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:37.498614 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:37.499040 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:37.500800 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:37.502305 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:37.503786 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:37.504606 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:37.504872 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:37.509623 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:37.509632 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:37.509649 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:37.509694 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:37.513200 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:37.514071 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:37.514078 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:37.514089 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:37.514264 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:37.514737 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:37.514749 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:37.514752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:37.515359 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:37.515704 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:37.516173 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 03:00:37.516635 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":428,"threads_started":1}
I20260808 03:00:37.518173 21175 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:00:37.518366 21175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:37.520825 21175 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:37.521418 21175 leader_election.cc:302] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa049867d0664e22838a4d12f7e50dc3; no voters:
I20260808 03:00:37.522080 21175 leader_election.cc:288] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:00:37.522312 21183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:00:37.523123 21183 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 1 LEADER]: Becoming Leader. State: Replica: fa049867d0664e22838a4d12f7e50dc3, State: Running, Role: LEADER
I20260808 03:00:37.523774 21183 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:37.524992 21175 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:00:37.527318 21188 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa049867d0664e22838a4d12f7e50dc3. Latest consensus state: current_term: 1 leader_uuid: "fa049867d0664e22838a4d12f7e50dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } } }
I20260808 03:00:37.527767 21188 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:37.528393 21186 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa049867d0664e22838a4d12f7e50dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } } }
I20260808 03:00:37.528893 21186 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:37.534803 21195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:00:37.539790 21195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:00:37.549851 21195 catalog_manager.cc:1347] Generated new cluster ID: edd28047407044da82538fd1ebb1829c
I20260808 03:00:37.550006 21195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:00:37.583508 21195 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:00:37.585629 21195 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:00:37.595381 21195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Generated new TSK 0
I20260808 03:00:37.596549 21195 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 03:00:37.677860 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35547
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:38.045681 21214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:38.046150 21214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:38.046347 21214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:38.054714 21214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:38.054903 21214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:38.055100 21214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:00:38.066098 21214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:38.067560 21214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:38.069010 21214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:38.083524 21223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:38.083618 21225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:38.083592 21228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:38.084561 21214 server_base.cc:1034] running on GCE node
I20260808 03:00:38.085314 21214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:38.086447 21214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:38.087736 21214 hybrid_clock.cc:648] HybridClock initialized: now 1786158038087647 us; error 71 us; skew 500 ppm
I20260808 03:00:38.088112 21214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:38 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158038, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:38.091688 21214 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:00:38.093345 21214 webserver.cc:460] Webserver started at http://127.0.205.129:44825/ using document root <none> and password file <none>
I20260808 03:00:38.094146 21214 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:38.094310 21214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:38.094663 21214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:00:38.098050 21214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "3806e18f82f1475fad4d83c4841514b9"
format_stamp: "Formatted at 2026-08-08 03:00:38 on dist-test-slave-6ntk"
server_key: "9d4af750b95b416e3184e2bc7782ef6d"
server_key_iv: "182fe700636d36d258c81357c047678b"
server_key_version: "encryptionkey@0"
I20260808 03:00:38.098875 21214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "3806e18f82f1475fad4d83c4841514b9"
format_stamp: "Formatted at 2026-08-08 03:00:38 on dist-test-slave-6ntk"
server_key: "9d4af750b95b416e3184e2bc7782ef6d"
server_key_iv: "182fe700636d36d258c81357c047678b"
server_key_version: "encryptionkey@0"
I20260808 03:00:38.104625 21214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 03:00:38.108603 21236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:38.110263 21214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:00:38.110425 21214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "3806e18f82f1475fad4d83c4841514b9"
format_stamp: "Formatted at 2026-08-08 03:00:38 on dist-test-slave-6ntk"
server_key: "9d4af750b95b416e3184e2bc7782ef6d"
server_key_iv: "182fe700636d36d258c81357c047678b"
server_key_version: "encryptionkey@0"
I20260808 03:00:38.110651 21214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:38.135505 21214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:38.140306 21214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:38.140573 21214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:38.141685 21214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:38.143239 21214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:38.143316 21214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:38.143404 21214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:38.143447 21214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:38.178593 21214 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:43121
I20260808 03:00:38.178637 21405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:43121 every 8 connection(s)
I20260808 03:00:38.179945 21214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:00:38.189033 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 21214
I20260808 03:00:38.189270 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 03:00:38.190186 822 external_mini_cluster.cc:1428] Setting key b760dd7a93716b441baec8965da8c547
Aug 08 03:00:38 dist-test-slave-6ntk krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158038, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:38.205128 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:38.182587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:38285 (local address 127.0.205.190:35547)
0808 03:00:38.183082 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:38.183087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:38.183906 (+ 819us) server_negotiation.cc:407] Connection header received
0808 03:00:38.186139 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:38.186145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:38.186289 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:38.186532 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:38.188908 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:38.190104 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:38.192540 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:38.193149 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:38.193299 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:38.197152 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:38.197162 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:38.197170 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:38.197218 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:38.201398 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:38.202378 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:38.202387 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:38.202394 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:38.202573 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:38.203040 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:38.203048 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:38.203054 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:38.203495 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:38.203696 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:38.204648 (+ 952us) server_negotiation.cc:299] Negotiation successful
0808 03:00:38.204866 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":264,"threads_started":1}
I20260808 03:00:38.206796 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:38.183004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:38285)
0808 03:00:38.183717 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:38.183766 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:38.185718 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:38.186816 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:38.186842 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:38.187698 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:38.188656 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:38.188686 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:38.190339 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:38.190346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:38.192331 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:38.192349 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:38.193405 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:38.193411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:38.193621 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:38.194427 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:38.194451 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:38.196867 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:38.201600 (+ 4733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:38.201609 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:38.201628 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:38.202234 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:38.202696 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:38.202700 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:38.202702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:38.202910 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:38.203631 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:38.203638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:38.204168 (+ 530us) client_negotiation.cc:769] Sending connection context
0808 03:00:38.204460 (+ 292us) client_negotiation.cc:240] Negotiation successful
0808 03:00:38.205139 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":279,"threads_started":1}
I20260808 03:00:38.208555 21406 heartbeater.cc:347] Connected to a master server at 127.0.205.190:35547
I20260808 03:00:38.208905 21406 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:38.209688 21406 heartbeater.cc:511] Master 127.0.205.190:35547 requested a full tablet report, sending...
I20260808 03:00:38.212153 21126 ts_manager.cc:194] Registered new tserver with Master: 3806e18f82f1475fad4d83c4841514b9 (127.0.205.129:43121)
I20260808 03:00:38.213660 21126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:38285
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 03:00:38.257517 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35547
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:38.616291 21416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:38.616667 21416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:38.616845 21416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:38.624281 21416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:38.624464 21416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:38.624655 21416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:00:38.634891 21416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:38.636303 21416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:38.637658 21416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:38.651649 21425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:38.651615 21426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:38.651544 21428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:38.652544 21416 server_base.cc:1034] running on GCE node
I20260808 03:00:38.653228 21416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:38.654318 21416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:38.655553 21416 hybrid_clock.cc:648] HybridClock initialized: now 1786158038655500 us; error 49 us; skew 500 ppm
I20260808 03:00:38.655912 21416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:38 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158038, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:38.659283 21416 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:00:38.660851 21416 webserver.cc:460] Webserver started at http://127.0.205.130:44969/ using document root <none> and password file <none>
I20260808 03:00:38.661559 21416 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:38.661711 21416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:38.662096 21416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:00:38.665078 21416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "181eb68089d94205961cde087fb6cc09"
format_stamp: "Formatted at 2026-08-08 03:00:38 on dist-test-slave-6ntk"
server_key: "d6bd487298ab60f77181d3d3e13c8e3d"
server_key_iv: "60dbf4ad6b9732844eed2f16463bad52"
server_key_version: "encryptionkey@0"
I20260808 03:00:38.666016 21416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "181eb68089d94205961cde087fb6cc09"
format_stamp: "Formatted at 2026-08-08 03:00:38 on dist-test-slave-6ntk"
server_key: "d6bd487298ab60f77181d3d3e13c8e3d"
server_key_iv: "60dbf4ad6b9732844eed2f16463bad52"
server_key_version: "encryptionkey@0"
I20260808 03:00:38.671355 21416 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 03:00:38.674811 21437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:38.676219 21416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:00:38.676409 21416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "181eb68089d94205961cde087fb6cc09"
format_stamp: "Formatted at 2026-08-08 03:00:38 on dist-test-slave-6ntk"
server_key: "d6bd487298ab60f77181d3d3e13c8e3d"
server_key_iv: "60dbf4ad6b9732844eed2f16463bad52"
server_key_version: "encryptionkey@0"
I20260808 03:00:38.676620 21416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:38.691661 21416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:38.696378 21416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:38.696710 21416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:38.697736 21416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:38.699198 21416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:38.699307 21416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:38.699460 21416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:38.699550 21416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:38.735095 21416 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:34615
I20260808 03:00:38.735143 21602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:34615 every 8 connection(s)
I20260808 03:00:38.736496 21416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:00:38.736960 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 21416
I20260808 03:00:38.737170 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 03:00:38.737831 822 external_mini_cluster.cc:1428] Setting key fc976258b2814add5babf9f9cb16a417
Aug 08 03:00:38 dist-test-slave-6ntk krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158038, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:38.761796 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:38.739431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:52707 (local address 127.0.205.190:35547)
0808 03:00:38.739993 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:38.740000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:38.741177 (+ 1177us) server_negotiation.cc:407] Connection header received
0808 03:00:38.744332 (+ 3155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:38.744337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:38.744426 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:38.744517 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:38.746895 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:38.747666 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:38.750275 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:38.750714 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:38.750813 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:38.755382 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:38.755388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:38.755393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:38.755431 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:38.758286 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:38.759274 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:38.759278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:38.759281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:38.759406 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:38.759957 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:38.759960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:38.759962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:38.760229 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:38.760354 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:38.761339 (+ 985us) server_negotiation.cc:299] Negotiation successful
0808 03:00:38.761516 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":243,"threads_started":1}
I20260808 03:00:38.763454 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:38.740073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:52707)
0808 03:00:38.740968 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:38.741024 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:38.743980 (+ 2956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:38.744837 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:38.744853 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:38.745718 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:38.746684 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:38.746714 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:38.747855 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:38.747863 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:38.750109 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:38.750129 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:38.750962 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:38.750971 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:38.751326 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:38.752124 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:38.752160 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:38.755097 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:38.758441 (+ 3344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:38.758454 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:38.758475 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:38.759145 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:38.759532 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:38.759540 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:38.759546 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:38.759816 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:38.760364 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:38.760374 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:38.760922 (+ 548us) client_negotiation.cc:769] Sending connection context
0808 03:00:38.761246 (+ 324us) client_negotiation.cc:240] Negotiation successful
0808 03:00:38.761863 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":338,"threads_started":1}
I20260808 03:00:38.765126 21604 heartbeater.cc:347] Connected to a master server at 127.0.205.190:35547
I20260808 03:00:38.765497 21604 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:38.766458 21604 heartbeater.cc:511] Master 127.0.205.190:35547 requested a full tablet report, sending...
I20260808 03:00:38.768054 21126 ts_manager.cc:194] Registered new tserver with Master: 181eb68089d94205961cde087fb6cc09 (127.0.205.130:34615)
I20260808 03:00:38.768925 21126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:52707
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 03:00:38.809842 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35547
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:39.163897 21617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:39.164207 21617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:39.164306 21617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:39.172298 21617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:39.172433 21617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:39.172551 21617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:00:39.185523 21617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:39.187054 21617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:39.188476 21617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:39.202466 21626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:39.202500 21624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:39.202538 21623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:39.203590 21617 server_base.cc:1034] running on GCE node
I20260808 03:00:39.204406 21617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:39.205503 21617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:39.206746 21617 hybrid_clock.cc:648] HybridClock initialized: now 1786158039206684 us; error 85 us; skew 500 ppm
I20260808 03:00:39.207099 21617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:39 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158039, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:39.210705 21617 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:00:39.212477 21617 webserver.cc:460] Webserver started at http://127.0.205.131:44401/ using document root <none> and password file <none>
I20260808 03:00:39.213233 21617 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:39.213400 21617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:39.213726 21617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:00:39.216990 21406 heartbeater.cc:503] Master 127.0.205.190:35547 was elected leader, sending a full tablet report...
I20260808 03:00:39.217108 21617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "e31ec108b1ea4b1e80f10cffde040ad1"
format_stamp: "Formatted at 2026-08-08 03:00:39 on dist-test-slave-6ntk"
server_key: "f63ddda69036083232efa6a92d538238"
server_key_iv: "6a158b3dfe0da0ce5bd28eabedecc561"
server_key_version: "encryptionkey@0"
I20260808 03:00:39.218205 21617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "e31ec108b1ea4b1e80f10cffde040ad1"
format_stamp: "Formatted at 2026-08-08 03:00:39 on dist-test-slave-6ntk"
server_key: "f63ddda69036083232efa6a92d538238"
server_key_iv: "6a158b3dfe0da0ce5bd28eabedecc561"
server_key_version: "encryptionkey@0"
I20260808 03:00:39.223821 21617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:00:39.227690 21640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:39.229077 21617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:00:39.229262 21617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "e31ec108b1ea4b1e80f10cffde040ad1"
format_stamp: "Formatted at 2026-08-08 03:00:39 on dist-test-slave-6ntk"
server_key: "f63ddda69036083232efa6a92d538238"
server_key_iv: "6a158b3dfe0da0ce5bd28eabedecc561"
server_key_version: "encryptionkey@0"
I20260808 03:00:39.229491 21617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:39.241047 21617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:39.245263 21617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:39.245577 21617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:39.246692 21617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:39.248039 21617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:39.248147 21617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:39.248306 21617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:39.248387 21617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:39.283699 21617 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:41689
I20260808 03:00:39.283735 21824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:41689 every 8 connection(s)
I20260808 03:00:39.285084 21617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:00:39.290012 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 21617
I20260808 03:00:39.290251 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 03:00:39.290884 822 external_mini_cluster.cc:1428] Setting key dc17f78cba1c221818c58c830779a812
Aug 08 03:00:39 dist-test-slave-6ntk krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158039, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:39.308557 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.287860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:55921 (local address 127.0.205.190:35547)
0808 03:00:39.288333 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:39.288338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:39.289650 (+ 1312us) server_negotiation.cc:407] Connection header received
0808 03:00:39.292512 (+ 2862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:39.292520 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:39.292618 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:39.292740 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:39.295146 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.295956 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.297622 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.298107 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.298210 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.301813 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:39.301818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:39.301822 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:39.301943 (+ 121us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:39.304876 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:39.305778 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:39.305784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:39.305787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:39.305994 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:39.306587 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:39.306594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:39.306596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:39.306988 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:39.307161 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:39.308101 (+ 940us) server_negotiation.cc:299] Negotiation successful
0808 03:00:39.308284 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":232,"threads_started":1}
I20260808 03:00:39.309556 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.288491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:55921)
0808 03:00:39.289385 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:39.289480 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:39.291962 (+ 2482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:39.292995 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:39.293005 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:39.293983 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:39.294969 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.294993 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.296197 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.296202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.297479 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.297496 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.298312 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.298317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.298544 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.299225 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:39.299248 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:39.301571 (+ 2323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:39.305048 (+ 3477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:39.305055 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:39.305070 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:39.305656 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:39.306122 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:39.306127 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:39.306130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:39.306455 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:39.307185 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:39.307193 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:39.307712 (+ 519us) client_negotiation.cc:769] Sending connection context
0808 03:00:39.307996 (+ 284us) client_negotiation.cc:240] Negotiation successful
0808 03:00:39.308431 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":319,"threads_started":1}
I20260808 03:00:39.310952 21825 heartbeater.cc:347] Connected to a master server at 127.0.205.190:35547
I20260808 03:00:39.311285 21825 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:39.312037 21825 heartbeater.cc:511] Master 127.0.205.190:35547 requested a full tablet report, sending...
I20260808 03:00:39.313417 21126 ts_manager.cc:194] Registered new tserver with Master: e31ec108b1ea4b1e80f10cffde040ad1 (127.0.205.131:41689)
I20260808 03:00:39.314339 21126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:55921
I20260808 03:00:39.319913 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:00:39.338634 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.324523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.0.205.190:35547)
0808 03:00:39.324798 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:39.324808 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:39.325149 (+ 341us) server_negotiation.cc:407] Connection header received
0808 03:00:39.325289 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:39.325292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:39.325396 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:39.325480 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:39.327178 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.328071 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.329714 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.330305 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.330414 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.333138 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:39.333144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:39.333147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:39.333241 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:39.336068 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:39.336723 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:39.336727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:39.336729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:39.336858 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:39.337196 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:39.337199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:39.337201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:39.337546 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:39.337738 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:39.338081 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 03:00:39.338263 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 03:00:39.343048 21126 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:00:39.350195 21126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:00:39.394855 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.363384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43121 (local address 127.0.0.1:59116)
0808 03:00:39.385373 (+ 21989us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:39.385413 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:39.385601 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:39.387132 (+ 1531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:39.387141 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:39.387175 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:39.387621 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.387635 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.389647 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.389653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.391737 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.391750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.393176 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.393182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.393920 (+ 738us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.394024 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 03:00:39.394180 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 03:00:39.394407 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21756,"mutex_wait_us":429,"thread_start_us":23201,"threads_started":1}
I20260808 03:00:39.395922 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.363982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.0.205.129:43121)
0808 03:00:39.385962 (+ 21980us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:39.385973 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:39.386007 (+ 34us) server_negotiation.cc:407] Connection header received
0808 03:00:39.386237 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:39.386248 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:39.386559 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:39.386817 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:39.387830 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.389470 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.391915 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.392921 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.393095 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.393371 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:39.395364 (+ 1993us) server_negotiation.cc:299] Negotiation successful
0808 03:00:39.395558 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21693,"thread_start_us":22205,"threads_started":1}
I20260808 03:00:39.396806 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.363050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:34615 (local address 127.0.0.1:54434)
0808 03:00:39.379143 (+ 16093us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:39.379211 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:39.379470 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:39.388235 (+ 8765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:39.388244 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:39.388264 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:39.388744 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.388755 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.390843 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.390856 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.393990 (+ 3134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.394012 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.396092 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.396098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.396281 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.396306 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 03:00:39.396364 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 03:00:39.396509 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15766,"mutex_wait_us":293,"thread_start_us":14845,"threads_started":1}
I20260808 03:00:39.397691 21849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.363307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54434 (local address 127.0.205.130:34615)
0808 03:00:39.387595 (+ 24288us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:39.387604 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:39.387633 (+ 29us) server_negotiation.cc:407] Connection header received
0808 03:00:39.387747 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:39.387755 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:39.387992 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:39.388849 (+ 857us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:39.389083 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.390634 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.394181 (+ 3547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.395839 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.397074 (+ 1235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.397234 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:39.397299 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 03:00:39.397422 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24110,"thread_start_us":23960,"threads_started":1}
I20260808 03:00:39.399128 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.366454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34084 (local address 127.0.205.131:41689)
0808 03:00:39.387406 (+ 20952us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:39.387415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:39.387451 (+ 36us) server_negotiation.cc:407] Connection header received
0808 03:00:39.387552 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:39.387557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:39.387780 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:39.388011 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:39.389111 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.390591 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.394618 (+ 4027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.397954 (+ 3336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.398113 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.398348 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:39.398572 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 03:00:39.398755 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20777,"thread_start_us":20588,"threads_started":1}
I20260808 03:00:39.400274 21316 tablet_service.cc:1467] Processing CreateTablet for tablet 01a9868ec81a465e8a6d5948ed60ff71 (DEFAULT_TABLE table=test-table [id=a22c6146483d455c87675aa0d607bdca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:00:39.401862 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.366081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:41689 (local address 127.0.0.1:34084)
0808 03:00:39.386907 (+ 20826us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:39.386928 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:39.387035 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:39.388011 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:39.388019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:39.388048 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:39.388533 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.388549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.390740 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.390747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.394416 (+ 3669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.394429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.398185 (+ 3756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.398192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.398371 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.398406 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 03:00:39.398508 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 03:00:39.401376 (+ 2868us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20671,"mutex_wait_us":495,"thread_start_us":18840,"threads_started":1}
I20260808 03:00:39.402185 21316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01a9868ec81a465e8a6d5948ed60ff71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:39.401724 21506 tablet_service.cc:1467] Processing CreateTablet for tablet 01a9868ec81a465e8a6d5948ed60ff71 (DEFAULT_TABLE table=test-table [id=a22c6146483d455c87675aa0d607bdca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:00:39.403342 21506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01a9868ec81a465e8a6d5948ed60ff71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:39.405473 21722 tablet_service.cc:1467] Processing CreateTablet for tablet 01a9868ec81a465e8a6d5948ed60ff71 (DEFAULT_TABLE table=test-table [id=a22c6146483d455c87675aa0d607bdca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:00:39.406903 21722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01a9868ec81a465e8a6d5948ed60ff71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:39.418712 21857 tablet_bootstrap.cc:492] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: Bootstrap starting.
I20260808 03:00:39.419363 21855 tablet_bootstrap.cc:492] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: Bootstrap starting.
I20260808 03:00:39.420266 21858 tablet_bootstrap.cc:492] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: Bootstrap starting.
I20260808 03:00:39.422991 21855 tablet_bootstrap.cc:654] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:39.423456 21858 tablet_bootstrap.cc:654] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:39.424098 21855 log.cc:824] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:39.424546 21858 log.cc:824] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:39.427507 21858 tablet_bootstrap.cc:492] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: No bootstrap required, opened a new log
I20260808 03:00:39.427666 21857 tablet_bootstrap.cc:654] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:39.427809 21858 ts_tablet_manager.cc:1397] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260808 03:00:39.428895 21857 log.cc:824] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:39.430320 21855 tablet_bootstrap.cc:492] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: No bootstrap required, opened a new log
I20260808 03:00:39.430593 21855 ts_tablet_manager.cc:1397] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260808 03:00:39.432114 21857 tablet_bootstrap.cc:492] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: No bootstrap required, opened a new log
I20260808 03:00:39.432459 21857 ts_tablet_manager.cc:1397] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260808 03:00:39.438794 21855 raft_consensus.cc:348] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:39.438815 21858 raft_consensus.cc:348] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:39.439205 21855 raft_consensus.cc:374] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:39.439316 21858 raft_consensus.cc:374] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:39.439337 21855 raft_consensus.cc:729] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181eb68089d94205961cde087fb6cc09, State: Initialized, Role: FOLLOWER
I20260808 03:00:39.439435 21858 raft_consensus.cc:729] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31ec108b1ea4b1e80f10cffde040ad1, State: Initialized, Role: FOLLOWER
I20260808 03:00:39.440026 21855 consensus_queue.cc:260] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:39.440050 21858 consensus_queue.cc:260] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:39.441250 21604 heartbeater.cc:503] Master 127.0.205.190:35547 was elected leader, sending a full tablet report...
I20260808 03:00:39.441992 21855 ts_tablet_manager.cc:1428] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260808 03:00:39.444494 21857 raft_consensus.cc:348] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:39.445084 21857 raft_consensus.cc:374] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:39.445217 21857 raft_consensus.cc:729] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3806e18f82f1475fad4d83c4841514b9, State: Initialized, Role: FOLLOWER
I20260808 03:00:39.446053 21857 consensus_queue.cc:260] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:39.446666 21825 heartbeater.cc:503] Master 127.0.205.190:35547 was elected leader, sending a full tablet report...
I20260808 03:00:39.448311 21857 ts_tablet_manager.cc:1428] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260808 03:00:39.448895 21858 ts_tablet_manager.cc:1428] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: Time spent starting tablet: real 0.021s user 0.017s sys 0.003s
W20260808 03:00:39.488806 21605 tablet.cc:2369] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:00:39.536890 21826 tablet.cc:2369] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:00:39.563894 21867 raft_consensus.cc:482] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:00:39.564218 21867 raft_consensus.cc:504] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:39.565999 21867 leader_election.cc:288] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 181eb68089d94205961cde087fb6cc09 (127.0.205.130:34615), 3806e18f82f1475fad4d83c4841514b9 (127.0.205.129:43121)
I20260808 03:00:39.573773 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.566542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:58663 (local address 127.0.205.129:43121)
0808 03:00:39.566772 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:39.566780 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:39.566852 (+ 72us) server_negotiation.cc:407] Connection header received
0808 03:00:39.567140 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:39.567145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:39.567228 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:39.567322 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:39.567899 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.569073 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.571590 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.572868 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.573042 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.573128 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:39.573404 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 03:00:39.573504 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 03:00:39.573925 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.566402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43121 (local address 127.0.205.131:58663)
0808 03:00:39.566750 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:39.566773 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:39.566940 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:39.567416 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:39.567423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:39.567458 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:39.567735 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.567745 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.569212 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.569216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.571407 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.571423 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.573024 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.573028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.573220 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.573261 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 03:00:39.573369 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 03:00:39.573515 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":67,"spinlock_wait_cycles":25344}
I20260808 03:00:39.574038 21849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.566506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:41437 (local address 127.0.205.130:34615)
0808 03:00:39.566833 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:39.566841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:39.567516 (+ 675us) server_negotiation.cc:407] Connection header received
0808 03:00:39.567742 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:39.567746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:39.567851 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:39.567967 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:39.568460 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.569417 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.571810 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.573059 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.573217 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.573266 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:39.573607 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 03:00:39.573729 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260808 03:00:39.574236 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.566613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:34615 (local address 127.0.205.131:41437)
0808 03:00:39.567399 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:39.567415 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:39.567549 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:39.567991 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:39.567994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:39.568014 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:39.568299 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.568306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.569566 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.569577 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.571651 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:39.571662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.573256 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.573261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:39.573437 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.573477 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 03:00:39.573576 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 03:00:39.573703 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":4,"thread_start_us":424,"threads_started":1}
I20260808 03:00:39.575304 21346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01a9868ec81a465e8a6d5948ed60ff71" candidate_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3806e18f82f1475fad4d83c4841514b9" is_pre_election: true
I20260808 03:00:39.575363 21537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01a9868ec81a465e8a6d5948ed60ff71" candidate_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181eb68089d94205961cde087fb6cc09" is_pre_election: true
I20260808 03:00:39.575857 21537 raft_consensus.cc:2454] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e31ec108b1ea4b1e80f10cffde040ad1 in term 0.
I20260808 03:00:39.575856 21346 raft_consensus.cc:2454] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e31ec108b1ea4b1e80f10cffde040ad1 in term 0.
I20260808 03:00:39.576608 21650 leader_election.cc:302] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181eb68089d94205961cde087fb6cc09, e31ec108b1ea4b1e80f10cffde040ad1; no voters:
I20260808 03:00:39.577153 21867 raft_consensus.cc:2781] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:00:39.577322 21867 raft_consensus.cc:482] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:00:39.577455 21867 raft_consensus.cc:3037] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:39.579794 21867 raft_consensus.cc:504] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:39.580499 21867 leader_election.cc:288] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [CANDIDATE]: Term 1 election: Requested vote from peers 181eb68089d94205961cde087fb6cc09 (127.0.205.130:34615), 3806e18f82f1475fad4d83c4841514b9 (127.0.205.129:43121)
I20260808 03:00:39.581007 21346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01a9868ec81a465e8a6d5948ed60ff71" candidate_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3806e18f82f1475fad4d83c4841514b9"
I20260808 03:00:39.581305 21537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01a9868ec81a465e8a6d5948ed60ff71" candidate_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181eb68089d94205961cde087fb6cc09"
I20260808 03:00:39.581339 21346 raft_consensus.cc:3037] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:39.581554 21537 raft_consensus.cc:3037] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:39.583806 21346 raft_consensus.cc:2454] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e31ec108b1ea4b1e80f10cffde040ad1 in term 1.
I20260808 03:00:39.583860 21537 raft_consensus.cc:2454] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e31ec108b1ea4b1e80f10cffde040ad1 in term 1.
I20260808 03:00:39.584281 21641 leader_election.cc:302] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3806e18f82f1475fad4d83c4841514b9, e31ec108b1ea4b1e80f10cffde040ad1; no voters:
I20260808 03:00:39.584700 21867 raft_consensus.cc:2781] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:00:39.585163 21867 raft_consensus.cc:686] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 1 LEADER]: Becoming Leader. State: Replica: e31ec108b1ea4b1e80f10cffde040ad1, State: Running, Role: LEADER
I20260808 03:00:39.585660 21867 consensus_queue.cc:237] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:39.590462 21126 catalog_manager.cc:5495] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e31ec108b1ea4b1e80f10cffde040ad1 (127.0.205.131). New cstate: current_term: 1 leader_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } health_report { overall_health: HEALTHY } } }
I20260808 03:00:39.637390 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.631455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.0.205.131:41689)
0808 03:00:39.631654 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:39.631659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:39.631733 (+ 74us) server_negotiation.cc:407] Connection header received
0808 03:00:39.631927 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:39.631931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:39.632012 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:39.632155 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:00:39.632602 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.633366 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.635301 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.635703 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.635804 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.636290 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:00:39.636849 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:00:39.636965 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:39.637104 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 03:00:39.637177 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:00:39.646018 21346 raft_consensus.cc:1261] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9 [term 1 FOLLOWER]: Refusing update from remote peer e31ec108b1ea4b1e80f10cffde040ad1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:00:39.646026 21537 raft_consensus.cc:1261] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09 [term 1 FOLLOWER]: Refusing update from remote peer e31ec108b1ea4b1e80f10cffde040ad1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:00:39.647080 21874 consensus_queue.cc:1035] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:00:39.647418 21867 consensus_queue.cc:1035] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:00:39.661861 21879 mvcc.cc:205] Tried to move back new op lower bound from 7316103330379272192 to 7316103330146938880. Current Snapshot: MvccSnapshot[applied={T|T < 7316103330379272192}]
I20260808 03:00:39.676195 21126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:00:39.676491 21126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:00:39.680440 21126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Sending DeleteTablet for 3 replicas of tablet 01a9868ec81a465e8a6d5948ed60ff71
I20260808 03:00:39.681838 21316 tablet_service.cc:1514] Processing DeleteTablet for tablet 01a9868ec81a465e8a6d5948ed60ff71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:00:39 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:59116
I20260808 03:00:39.682046 21722 tablet_service.cc:1514] Processing DeleteTablet for tablet 01a9868ec81a465e8a6d5948ed60ff71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:00:39 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:34084
I20260808 03:00:39.682197 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 21214
I20260808 03:00:39.683113 21506 tablet_service.cc:1514] Processing DeleteTablet for tablet 01a9868ec81a465e8a6d5948ed60ff71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:00:39 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:54434
I20260808 03:00:39.693297 21896 tablet_replica.cc:330] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: stopping tablet replica
I20260808 03:00:39.694681 21896 raft_consensus.cc:2229] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:00:39.694748 21895 tablet_replica.cc:330] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: stopping tablet replica
I20260808 03:00:39.695266 21895 raft_consensus.cc:2229] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:00:39.696115 21895 raft_consensus.cc:2258] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:00:39.696645 21896 raft_consensus.cc:2258] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:00:39.696756 21898 tablet_replica.cc:330] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: stopping tablet replica
I20260808 03:00:39.699514 21895 ts_tablet_manager.cc:1905] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:00:39.697474 21898 raft_consensus.cc:2229] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:00:39.699687 21896 ts_tablet_manager.cc:1905] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:00:39.700217 21898 raft_consensus.cc:2258] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:00:39.702920 21898 ts_tablet_manager.cc:1905] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:00:39.708213 21895 ts_tablet_manager.cc:1918] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:00:39.708488 21895 log.cc:1196] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/01a9868ec81a465e8a6d5948ed60ff71
Aug 08 03:00:39 dist-test-slave-6ntk krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 03:00:39.709259 21895 ts_tablet_manager.cc:1939] T 01a9868ec81a465e8a6d5948ed60ff71 P 3806e18f82f1475fad4d83c4841514b9: Deleting consensus metadata
I20260808 03:00:39.710624 21896 ts_tablet_manager.cc:1918] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:00:39.711609 21107 catalog_manager.cc:4841] TS 3806e18f82f1475fad4d83c4841514b9 (127.0.205.129:43121): tablet 01a9868ec81a465e8a6d5948ed60ff71 (table test-table [id=a22c6146483d455c87675aa0d607bdca]) successfully deleted
I20260808 03:00:39.712342 21896 log.cc:1196] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/01a9868ec81a465e8a6d5948ed60ff71
I20260808 03:00:39.713114 21896 ts_tablet_manager.cc:1939] T 01a9868ec81a465e8a6d5948ed60ff71 P e31ec108b1ea4b1e80f10cffde040ad1: Deleting consensus metadata
I20260808 03:00:39.715319 21110 catalog_manager.cc:4841] TS e31ec108b1ea4b1e80f10cffde040ad1 (127.0.205.131:41689): tablet 01a9868ec81a465e8a6d5948ed60ff71 (table test-table [id=a22c6146483d455c87675aa0d607bdca]) successfully deleted
I20260808 03:00:39.717914 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.683495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.0.205.129:43121)
0808 03:00:39.683672 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:39.683678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:39.695734 (+ 12056us) server_negotiation.cc:407] Connection header received
0808 03:00:39.697554 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:39.697561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:39.697650 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:39.697773 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:39.699949 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.701037 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.703189 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.703858 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.703958 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.710028 (+ 6070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:39.710040 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:39.710059 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:39.710108 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:39.714434 (+ 4326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:39.715514 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:39.715523 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:39.715526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:39.715710 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:39.716198 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:39.716203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:39.716206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:39.716611 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:39.716941 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:39.717273 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 03:00:39.717496 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 03:00:39.718554 21400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:39.718828 21898 ts_tablet_manager.cc:1918] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:00:39.719120 21898 log.cc:1196] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/01a9868ec81a465e8a6d5948ed60ff71
I20260808 03:00:39.720137 21898 ts_tablet_manager.cc:1939] T 01a9868ec81a465e8a6d5948ed60ff71 P 181eb68089d94205961cde087fb6cc09: Deleting consensus metadata
I20260808 03:00:39.722441 21111 catalog_manager.cc:4841] TS 181eb68089d94205961cde087fb6cc09 (127.0.205.130:34615): tablet 01a9868ec81a465e8a6d5948ed60ff71 (table test-table [id=a22c6146483d455c87675aa0d607bdca]) successfully deleted
I20260808 03:00:39.831314 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 21214
I20260808 03:00:39.861310 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 21416
Aug 08 03:00:39 dist-test-slave-6ntk krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 03:00:39.880208 21849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:39.862323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54448 (local address 127.0.205.130:34615)
0808 03:00:39.862575 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:39.862583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:39.862612 (+ 29us) server_negotiation.cc:407] Connection header received
0808 03:00:39.862696 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:39.862703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:39.862797 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:39.862931 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:39.864221 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.865417 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.867030 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:39.867703 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:39.867848 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:39.872627 (+ 4779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:39.872650 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:39.872672 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:39.872729 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:39.876881 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:39.877642 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:39.877653 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:39.877657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:39.877832 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:39.878357 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:39.878363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:39.878367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:39.878943 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:39.879310 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:39.879505 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 03:00:39.879747 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 03:00:39.881130 21594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:39.974642 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 21416
I20260808 03:00:40.003142 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 21617
Aug 08 03:00:40 dist-test-slave-6ntk krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 03:00:40.021271 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:40.004246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34110 (local address 127.0.205.131:41689)
0808 03:00:40.004468 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:40.004477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:40.004513 (+ 36us) server_negotiation.cc:407] Connection header received
0808 03:00:40.004592 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:40.004597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:40.004658 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:40.004786 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:40.006671 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:40.007541 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:40.008916 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:40.009399 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:40.009559 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:40.014717 (+ 5158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:40.014732 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:40.014749 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:40.014794 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:40.018079 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:40.018882 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:40.018896 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:40.018899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:40.019097 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:40.019512 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:40.019518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:40.019522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:40.019935 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:40.020278 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:40.020622 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 03:00:40.020855 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 03:00:40.022221 21819 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:40.151373 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 21617
I20260808 03:00:40.181957 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 21083
I20260808 03:00:40.183272 21166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:40.303188 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 21083
I20260808 03:00:40.334375 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:35547
--webserver_interface=127.0.205.190
--webserver_port=41229
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:35547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:40.694725 21907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:40.695109 21907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:40.695282 21907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:40.702741 21907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:00:40.702943 21907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:40.703059 21907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:40.703167 21907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:00:40.703250 21907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:00:40.714690 21907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:35547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:35547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=41229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:40.716145 21907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:40.717545 21907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:40.744969 21914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:40.745008 21916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:40.744904 21918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:40.746112 21907 server_base.cc:1034] running on GCE node
I20260808 03:00:40.747128 21907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:40.748860 21907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:40.750078 21907 hybrid_clock.cc:648] HybridClock initialized: now 1786158040750026 us; error 38 us; skew 500 ppm
I20260808 03:00:40.750550 21907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:40 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158040, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:40.760903 21907 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:00:40.763001 21907 webserver.cc:460] Webserver started at http://127.0.205.190:41229/ using document root <none> and password file <none>
I20260808 03:00:40.763921 21907 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:40.764093 21907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:40.771232 21907 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:00:40.774724 21927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:40.776261 21907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:00:40.776427 21907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "fa049867d0664e22838a4d12f7e50dc3"
format_stamp: "Formatted at 2026-08-08 03:00:37 on dist-test-slave-6ntk"
server_key: "71bb3a64ef4468bb335de202596cf082"
server_key_iv: "bc7ca34db324690da1ab79a970908704"
server_key_version: "encryptionkey@0"
I20260808 03:00:40.777531 21907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:40.798851 21907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:40.803416 21907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:40.803680 21907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:40.824638 21907 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:35547
I20260808 03:00:40.824659 22004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:35547 every 8 connection(s)
I20260808 03:00:40.826431 21907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:00:40.834005 22006 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:00:40.835901 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 21907
I20260808 03:00:40.836973 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:43121
--local_ip_for_outbound_sockets=127.0.205.129
--tserver_master_addrs=127.0.205.190:35547
--webserver_port=44825
--webserver_interface=127.0.205.129
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:00:40.838776 22006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Bootstrap starting.
I20260808 03:00:40.862725 22006 log.cc:824] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:40.875665 22006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:00:40.876259 22006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Bootstrap complete.
I20260808 03:00:40.886413 22006 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:40.887279 22006 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa049867d0664e22838a4d12f7e50dc3, State: Initialized, Role: FOLLOWER
I20260808 03:00:40.887965 22006 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:40.888221 22006 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:00:40.888412 22006 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:00:40.888612 22006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 1 FOLLOWER]: Advancing to term 2
I20260808 03:00:40.892786 22006 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:40.893496 22006 leader_election.cc:302] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa049867d0664e22838a4d12f7e50dc3; no voters:
I20260808 03:00:40.894459 22006 leader_election.cc:288] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 03:00:40.894722 22011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 03:00:40.898114 22011 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 2 LEADER]: Becoming Leader. State: Replica: fa049867d0664e22838a4d12f7e50dc3, State: Running, Role: LEADER
I20260808 03:00:40.898841 22011 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:40.899096 22006 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:00:40.902048 22017 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa049867d0664e22838a4d12f7e50dc3. Latest consensus state: current_term: 2 leader_uuid: "fa049867d0664e22838a4d12f7e50dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } } }
I20260808 03:00:40.902549 22017 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:40.903158 22012 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fa049867d0664e22838a4d12f7e50dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } } }
I20260808 03:00:40.903523 22012 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:40.908058 22021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:00:40.917712 22021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:00:40.919418 22021 catalog_manager.cc:1259] Loaded cluster ID: edd28047407044da82538fd1ebb1829c
I20260808 03:00:40.919601 22021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:00:40.923380 22021 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 03:00:40.941510 22021 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 03:00:40.951228 22021 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:00:40.953087 22021 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Loaded TSK: 0
I20260808 03:00:40.954720 22021 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 03:00:41.220995 22009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:41.221367 22009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:41.221550 22009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:41.229192 22009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:41.229360 22009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:41.229554 22009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:00:41.239732 22009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:43121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=44825
--tserver_master_addrs=127.0.205.190:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:41.241199 22009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:41.242614 22009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:41.256668 22047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:41.257227 22046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:41.257480 22051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:41.257912 22009 server_base.cc:1034] running on GCE node
I20260808 03:00:41.258783 22009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:41.260089 22009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:41.261354 22009 hybrid_clock.cc:648] HybridClock initialized: now 1786158041261288 us; error 49 us; skew 500 ppm
I20260808 03:00:41.261751 22009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:41 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:41.266191 22009 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:00:41.267989 22009 webserver.cc:460] Webserver started at http://127.0.205.129:44825/ using document root <none> and password file <none>
I20260808 03:00:41.268945 22009 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:41.269106 22009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:41.276301 22009 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:00:41.279992 22065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:41.281980 22009 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 03:00:41.282194 22009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "3806e18f82f1475fad4d83c4841514b9"
format_stamp: "Formatted at 2026-08-08 03:00:38 on dist-test-slave-6ntk"
server_key: "9d4af750b95b416e3184e2bc7782ef6d"
server_key_iv: "182fe700636d36d258c81357c047678b"
server_key_version: "encryptionkey@0"
I20260808 03:00:41.283466 22009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:41.302374 22009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:41.306713 22009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:41.307019 22009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:41.308089 22009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:41.309693 22009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:41.309840 22009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:41.310004 22009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:41.310093 22009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:41.344581 22009 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:43121
I20260808 03:00:41.344645 22225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:43121 every 8 connection(s)
I20260808 03:00:41.346249 22009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:00:41.352106 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 22009
I20260808 03:00:41.353143 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:34615
--local_ip_for_outbound_sockets=127.0.205.130
--tserver_master_addrs=127.0.205.190:35547
--webserver_port=44969
--webserver_interface=127.0.205.130
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:00:41 dist-test-slave-6ntk krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:41.375226 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:41.349472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:41469 (local address 127.0.205.190:35547)
0808 03:00:41.356083 (+ 6611us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:41.356095 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:41.356137 (+ 42us) server_negotiation.cc:407] Connection header received
0808 03:00:41.356451 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:41.356465 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:41.356809 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:41.357240 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:41.358973 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:41.360251 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:41.361851 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:41.362589 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:41.362809 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:41.366903 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:41.366912 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:41.366927 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:41.366966 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:41.370136 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:41.371250 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:41.371258 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:41.371264 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:41.371441 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:41.371978 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:41.371981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:41.371983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:41.372468 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:41.372625 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:41.373661 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0808 03:00:41.374108 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6424,"thread_start_us":6308,"threads_started":1}
I20260808 03:00:41.375530 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:41.349636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:41469)
0808 03:00:41.353245 (+ 3609us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:41.353304 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:41.355388 (+ 2084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:41.357402 (+ 2014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:41.357412 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:41.358122 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:41.358697 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:41.358717 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:41.360446 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:41.360451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:41.361688 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:41.361701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:41.362841 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:41.362848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:41.363138 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:41.363891 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:41.363915 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:41.366588 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:41.370333 (+ 3745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:41.370346 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:41.370366 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:41.371092 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:41.371564 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:41.371571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:41.371578 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:41.371849 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:41.372621 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:41.372632 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:41.373101 (+ 469us) client_negotiation.cc:769] Sending connection context
0808 03:00:41.373434 (+ 333us) client_negotiation.cc:240] Negotiation successful
0808 03:00:41.374124 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3270,"thread_start_us":3048,"threads_started":1}
I20260808 03:00:41.378912 22226 heartbeater.cc:347] Connected to a master server at 127.0.205.190:35547
I20260808 03:00:41.379285 22226 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:41.380468 22226 heartbeater.cc:511] Master 127.0.205.190:35547 requested a full tablet report, sending...
I20260808 03:00:41.383198 21948 ts_manager.cc:194] Registered new tserver with Master: 3806e18f82f1475fad4d83c4841514b9 (127.0.205.129:43121)
I20260808 03:00:41.384745 21948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:41469
W20260808 03:00:41.720829 22232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:41.721148 22232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:41.721254 22232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:41.728703 22232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:41.728824 22232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:41.728938 22232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:00:41.739188 22232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:34615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=44969
--tserver_master_addrs=127.0.205.190:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:41.740707 22232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:41.742089 22232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:41.756002 22241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:41.756059 22247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:41.756093 22242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:41.757200 22232 server_base.cc:1034] running on GCE node
I20260808 03:00:41.757808 22232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:41.758874 22232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:41.760087 22232 hybrid_clock.cc:648] HybridClock initialized: now 1786158041760036 us; error 38 us; skew 500 ppm
I20260808 03:00:41.760466 22232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:41 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:41.763944 22232 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:00:41.765761 22232 webserver.cc:460] Webserver started at http://127.0.205.130:44969/ using document root <none> and password file <none>
I20260808 03:00:41.766609 22232 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:41.766809 22232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:41.772816 22232 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 03:00:41.775980 22256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:41.777482 22232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:00:41.777691 22232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "181eb68089d94205961cde087fb6cc09"
format_stamp: "Formatted at 2026-08-08 03:00:38 on dist-test-slave-6ntk"
server_key: "d6bd487298ab60f77181d3d3e13c8e3d"
server_key_iv: "60dbf4ad6b9732844eed2f16463bad52"
server_key_version: "encryptionkey@0"
I20260808 03:00:41.778808 22232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:41.799232 22232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:41.804205 22232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:41.804463 22232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:41.805467 22232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:41.806967 22232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:41.807067 22232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:41.807161 22232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:41.807226 22232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:41.842967 22232 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:34615
I20260808 03:00:41.843008 22439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:34615 every 8 connection(s)
I20260808 03:00:41.844812 22232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:00:41.847245 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 22232
I20260808 03:00:41.847904 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:41689
--local_ip_for_outbound_sockets=127.0.205.131
--tserver_master_addrs=127.0.205.190:35547
--webserver_port=44401
--webserver_interface=127.0.205.131
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:00:41 dist-test-slave-6ntk krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:41.871464 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:41.854108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:49473 (local address 127.0.205.190:35547)
0808 03:00:41.854336 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:41.854345 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:41.854367 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:00:41.854430 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:41.854438 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:41.854559 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:41.854663 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:41.856890 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:41.857929 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:41.859868 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:41.860633 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:41.860776 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:41.864732 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:41.864737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:41.864740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:41.864773 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:41.867874 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:41.868943 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:41.868948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:41.868950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:41.869098 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:41.869621 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:41.869624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:41.869626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:41.869976 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:41.870094 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:41.871023 (+ 929us) server_negotiation.cc:299] Negotiation successful
0808 03:00:41.871200 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:00:41.873209 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:41.848428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:49473)
0808 03:00:41.849355 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:41.849432 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:41.852180 (+ 2748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:41.854977 (+ 2797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:41.854992 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:41.855822 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:41.856684 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:41.856714 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:41.858113 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:41.858121 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:41.859686 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:41.859699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:41.860861 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:41.860867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:41.861065 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:41.861782 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:41.861810 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:41.864494 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:41.868038 (+ 3544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:41.868050 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:41.868072 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:41.868807 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:41.869226 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:41.869233 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:41.869240 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:41.869503 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:41.870123 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:41.870135 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:41.870618 (+ 483us) client_negotiation.cc:769] Sending connection context
0808 03:00:41.870952 (+ 334us) client_negotiation.cc:240] Negotiation successful
0808 03:00:41.871556 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":332,"threads_started":1}
I20260808 03:00:41.874912 22440 heartbeater.cc:347] Connected to a master server at 127.0.205.190:35547
I20260808 03:00:41.875247 22440 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:41.876001 22440 heartbeater.cc:511] Master 127.0.205.190:35547 requested a full tablet report, sending...
I20260808 03:00:41.877638 21948 ts_manager.cc:194] Registered new tserver with Master: 181eb68089d94205961cde087fb6cc09 (127.0.205.130:34615)
I20260808 03:00:41.878500 21948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:49473
W20260808 03:00:42.205660 22444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:42.206135 22444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:42.206346 22444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:42.214118 22444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:42.214275 22444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:42.214471 22444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:00:42.224782 22444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:41689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=44401
--tserver_master_addrs=127.0.205.190:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:42.226264 22444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:42.227666 22444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:42.241703 22453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:42.241608 22450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:42.241631 22451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:42.242678 22444 server_base.cc:1034] running on GCE node
I20260808 03:00:42.243450 22444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:42.244478 22444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:42.245721 22444 hybrid_clock.cc:648] HybridClock initialized: now 1786158042245659 us; error 58 us; skew 500 ppm
I20260808 03:00:42.246140 22444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:42 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158042, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:42.249547 22444 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:00:42.251240 22444 webserver.cc:460] Webserver started at http://127.0.205.131:44401/ using document root <none> and password file <none>
I20260808 03:00:42.251991 22444 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:42.252164 22444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:42.257802 22444 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 03:00:42.260844 22467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:42.262380 22444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:00:42.262570 22444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "e31ec108b1ea4b1e80f10cffde040ad1"
format_stamp: "Formatted at 2026-08-08 03:00:39 on dist-test-slave-6ntk"
server_key: "f63ddda69036083232efa6a92d538238"
server_key_iv: "6a158b3dfe0da0ce5bd28eabedecc561"
server_key_version: "encryptionkey@0"
I20260808 03:00:42.263604 22444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:42.278795 22444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:42.283761 22444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:42.284082 22444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:42.285113 22444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:42.286839 22444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:42.286967 22444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:42.287137 22444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:42.287240 22444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:42.322947 22444 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:41689
I20260808 03:00:42.323045 22644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:41689 every 8 connection(s)
I20260808 03:00:42.324509 22444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:00:42.331451 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 22444
Aug 08 03:00:42 dist-test-slave-6ntk krb5kdc[21062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158042, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:00:42.345820 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.327241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:60715 (local address 127.0.205.190:35547)
0808 03:00:42.327424 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.327428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.328518 (+ 1090us) server_negotiation.cc:407] Connection header received
0808 03:00:42.330431 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.330436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.330535 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.330648 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:42.332324 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.333360 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.335148 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.335600 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.335753 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.339398 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:42.339404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:42.339407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:42.339440 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:42.342401 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:42.343183 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:42.343189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:42.343191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:42.343326 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:42.343920 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:42.343929 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:42.343932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:42.344313 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:42.344484 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.345233 (+ 749us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.345479 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:00:42.347038 22647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.327640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:60715)
0808 03:00:42.328341 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.328381 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:42.330155 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:42.330887 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:42.330898 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:42.331439 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:00:42.332117 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.332145 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.333551 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.333560 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.334994 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.335011 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.335854 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.335861 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.336116 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.336847 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:00:42.336870 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:00:42.339165 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:00:42.342560 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:42.342569 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:42.342588 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:42.343042 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:42.343496 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:00:42.343504 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:00:42.343508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:00:42.343751 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:00:42.344455 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:00:42.344464 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:00:42.344831 (+ 367us) client_negotiation.cc:769] Sending connection context
0808 03:00:42.345116 (+ 285us) client_negotiation.cc:240] Negotiation successful
0808 03:00:42.345697 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":281,"threads_started":1}
I20260808 03:00:42.348867 22645 heartbeater.cc:347] Connected to a master server at 127.0.205.190:35547
I20260808 03:00:42.349232 22645 heartbeater.cc:464] Registering TS with master...
I20260808 03:00:42.350205 22645 heartbeater.cc:511] Master 127.0.205.190:35547 requested a full tablet report, sending...
I20260808 03:00:42.350766 22648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.332508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50052 (local address 127.0.205.190:35547)
0808 03:00:42.333116 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.333123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.333985 (+ 862us) server_negotiation.cc:407] Connection header received
0808 03:00:42.334197 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.334203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.334260 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.334376 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:42.335941 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.336849 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.338343 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.338883 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.339024 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.342150 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:42.342155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:42.342158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:42.342199 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:42.345555 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:42.347092 (+ 1537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:42.347097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:42.347101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:42.347256 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:42.347927 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:42.347930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:42.347932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:42.348326 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:42.348612 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.350085 (+ 1473us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.350366 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":268,"threads_started":1}
I20260808 03:00:42.351958 21948 ts_manager.cc:194] Registered new tserver with Master: e31ec108b1ea4b1e80f10cffde040ad1 (127.0.205.131:41689)
I20260808 03:00:42.352862 21948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:60715
I20260808 03:00:42.353995 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:00:42.361088 22648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.354956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.0.205.190:35547)
0808 03:00:42.355173 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.355183 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.355647 (+ 464us) server_negotiation.cc:407] Connection header received
0808 03:00:42.355817 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.355821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.355950 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.356071 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:00:42.356486 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.357216 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.358750 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.359188 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.359306 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.359818 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:00:42.360457 (+ 639us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:00:42.360585 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.360674 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.360752 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:00:42.362216 21947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:00:42.370828 21947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:00:42.389988 22226 heartbeater.cc:503] Master 127.0.205.190:35547 was elected leader, sending a full tablet report...
I20260808 03:00:42.399132 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.384228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:34615 (local address 127.0.0.1:40882)
0808 03:00:42.386083 (+ 1855us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.386107 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:42.386247 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:42.388612 (+ 2365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:42.388621 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:42.388659 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:42.389185 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.389202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.392025 (+ 2823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.392035 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.395791 (+ 3756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.395821 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.397971 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.397977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.398192 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.398279 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 03:00:42.398434 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 03:00:42.398656 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1694,"mutex_wait_us":98,"thread_start_us":707,"threads_started":1}
I20260808 03:00:42.399643 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.384777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40882 (local address 127.0.205.130:34615)
0808 03:00:42.387535 (+ 2758us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.387543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.387574 (+ 31us) server_negotiation.cc:407] Connection header received
0808 03:00:42.387676 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.387686 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.388310 (+ 624us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.389445 (+ 1135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:42.389736 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.391803 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.396633 (+ 4830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.397710 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.397933 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.398181 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.399121 (+ 940us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.399310 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2589,"thread_start_us":2483,"threads_started":1}
I20260808 03:00:42.400177 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.384091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43121 (local address 127.0.0.1:49718)
0808 03:00:42.385038 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.385094 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:42.385274 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:42.386984 (+ 1710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:42.386992 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:42.387031 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:42.387590 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.387606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.392450 (+ 4844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.392455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.396218 (+ 3763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.396233 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.398818 (+ 2585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.398825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.399053 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.399090 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 03:00:42.399188 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 03:00:42.399318 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":515,"thread_start_us":1498,"threads_started":1}
I20260808 03:00:42.400424 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.384429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49718 (local address 127.0.205.129:43121)
0808 03:00:42.386416 (+ 1987us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.386423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.386448 (+ 25us) server_negotiation.cc:407] Connection header received
0808 03:00:42.386533 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.386538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.386766 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.386962 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:42.390211 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.392272 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.396451 (+ 4179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.398521 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.398672 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.399001 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.399841 (+ 840us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.400051 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1824,"thread_start_us":1671,"threads_started":1}
I20260808 03:00:42.404460 22337 tablet_service.cc:1467] Processing CreateTablet for tablet f1c4177045644179926f6d0173bdf44d (DEFAULT_TABLE table=test-table [id=6a79b225c41d4573944b16fab28d94f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:00:42.404460 22142 tablet_service.cc:1467] Processing CreateTablet for tablet f1c4177045644179926f6d0173bdf44d (DEFAULT_TABLE table=test-table [id=6a79b225c41d4573944b16fab28d94f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:00:42.405810 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.387493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:41689 (local address 127.0.0.1:34566)
0808 03:00:42.396069 (+ 8576us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.396090 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:42.396211 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:42.399463 (+ 3252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:42.399467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:42.399497 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:42.400049 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.400066 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.402343 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.402350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.403622 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.403635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.404953 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.404959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.405136 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.405181 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 03:00:42.405283 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 03:00:42.405425 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8423,"thread_start_us":8800,"threads_started":1}
I20260808 03:00:42.406149 22337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c4177045644179926f6d0173bdf44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:42.406169 22142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c4177045644179926f6d0173bdf44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:42.406244 22668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.387951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34566 (local address 127.0.205.131:41689)
0808 03:00:42.398718 (+ 10767us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.398727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.398772 (+ 45us) server_negotiation.cc:407] Connection header received
0808 03:00:42.398938 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.398947 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.399232 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.400311 (+ 1079us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:42.400596 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.402207 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.403807 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.404719 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.404890 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.405129 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.405383 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.405546 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10593,"thread_start_us":10429,"threads_started":1}
I20260808 03:00:42.410509 22540 tablet_service.cc:1467] Processing CreateTablet for tablet f1c4177045644179926f6d0173bdf44d (DEFAULT_TABLE table=test-table [id=6a79b225c41d4573944b16fab28d94f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:00:42.412122 22540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1c4177045644179926f6d0173bdf44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:00:42.419772 22669 tablet_bootstrap.cc:492] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: Bootstrap starting.
I20260808 03:00:42.422292 22670 tablet_bootstrap.cc:492] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Bootstrap starting.
I20260808 03:00:42.423969 22669 tablet_bootstrap.cc:654] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:42.425382 22669 log.cc:824] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:42.426110 22670 tablet_bootstrap.cc:654] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:42.427213 22670 log.cc:824] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:42.427659 22674 tablet_bootstrap.cc:492] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: Bootstrap starting.
I20260808 03:00:42.429255 22669 tablet_bootstrap.cc:492] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: No bootstrap required, opened a new log
I20260808 03:00:42.429538 22669 ts_tablet_manager.cc:1397] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260808 03:00:42.430595 22670 tablet_bootstrap.cc:492] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: No bootstrap required, opened a new log
I20260808 03:00:42.430886 22674 tablet_bootstrap.cc:654] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: Neither blocks nor log segments found. Creating new log.
I20260808 03:00:42.430881 22670 ts_tablet_manager.cc:1397] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260808 03:00:42.431869 22674 log.cc:824] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:42.434829 22674 tablet_bootstrap.cc:492] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: No bootstrap required, opened a new log
I20260808 03:00:42.435154 22674 ts_tablet_manager.cc:1397] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 03:00:42.440456 22669 raft_consensus.cc:348] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:42.440955 22669 raft_consensus.cc:374] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:42.441129 22669 raft_consensus.cc:729] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181eb68089d94205961cde087fb6cc09, State: Initialized, Role: FOLLOWER
I20260808 03:00:42.441500 22670 raft_consensus.cc:348] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:42.441988 22670 raft_consensus.cc:374] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:42.441835 22669 consensus_queue.cc:260] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:42.442093 22670 raft_consensus.cc:729] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3806e18f82f1475fad4d83c4841514b9, State: Initialized, Role: FOLLOWER
I20260808 03:00:42.442819 22670 consensus_queue.cc:260] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:42.442976 22674 raft_consensus.cc:348] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:42.443378 22674 raft_consensus.cc:374] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:00:42.443545 22674 raft_consensus.cc:729] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31ec108b1ea4b1e80f10cffde040ad1, State: Initialized, Role: FOLLOWER
I20260808 03:00:42.444272 22674 consensus_queue.cc:260] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:42.445127 22670 ts_tablet_manager.cc:1428] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260808 03:00:42.446185 22440 heartbeater.cc:503] Master 127.0.205.190:35547 was elected leader, sending a full tablet report...
I20260808 03:00:42.446832 22669 ts_tablet_manager.cc:1428] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: Time spent starting tablet: real 0.017s user 0.012s sys 0.005s
I20260808 03:00:42.447914 22645 heartbeater.cc:503] Master 127.0.205.190:35547 was elected leader, sending a full tablet report...
I20260808 03:00:42.448608 22674 ts_tablet_manager.cc:1428] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260808 03:00:42.476085 22680 raft_consensus.cc:482] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:00:42.476382 22680 raft_consensus.cc:504] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:42.478188 22680 leader_election.cc:288] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 181eb68089d94205961cde087fb6cc09 (127.0.205.130:34615), e31ec108b1ea4b1e80f10cffde040ad1 (127.0.205.131:41689)
I20260808 03:00:42.485332 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.478516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:40275 (local address 127.0.205.130:34615)
0808 03:00:42.478765 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.478775 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.479467 (+ 692us) server_negotiation.cc:407] Connection header received
0808 03:00:42.479716 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.479720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.479782 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.479877 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:42.480462 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.481267 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.483404 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.484301 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.484483 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.484567 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.484922 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.485057 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 03:00:42.485615 22668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.478788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:54441 (local address 127.0.205.131:41689)
0808 03:00:42.479039 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.479048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.479746 (+ 698us) server_negotiation.cc:407] Connection header received
0808 03:00:42.479956 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.479960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.480018 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.480122 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:00:42.480714 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.481571 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.483791 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.484696 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.484876 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.484951 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.485238 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.485364 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 03:00:42.485694 22684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.478640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:41689 (local address 127.0.205.129:54441)
0808 03:00:42.479636 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.479650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:42.479809 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:42.480143 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:42.480147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:42.480176 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:42.480545 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.480553 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.481708 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.481715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.483618 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.483630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.484903 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.484909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.485077 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.485100 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 03:00:42.485209 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 03:00:42.485345 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"thread_start_us":610,"threads_started":1}
I20260808 03:00:42.486231 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.478463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:34615 (local address 127.0.205.129:40275)
0808 03:00:42.479335 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.479351 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 03:00:42.479556 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:00:42.479895 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:00:42.479900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:00:42.479924 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:00:42.480310 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.480319 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.481443 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.481451 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.483241 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:00:42.483254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.484539 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.484545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:00:42.484692 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.484716 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 03:00:42.485797 (+ 1081us) client_negotiation.cc:240] Negotiation successful
0808 03:00:42.485960 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":514,"threads_started":1}
I20260808 03:00:42.486850 22569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c4177045644179926f6d0173bdf44d" candidate_uuid: "3806e18f82f1475fad4d83c4841514b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" is_pre_election: true
I20260808 03:00:42.487113 22365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c4177045644179926f6d0173bdf44d" candidate_uuid: "3806e18f82f1475fad4d83c4841514b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181eb68089d94205961cde087fb6cc09" is_pre_election: true
I20260808 03:00:42.487481 22569 raft_consensus.cc:2454] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3806e18f82f1475fad4d83c4841514b9 in term 0.
I20260808 03:00:42.487758 22365 raft_consensus.cc:2454] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3806e18f82f1475fad4d83c4841514b9 in term 0.
I20260808 03:00:42.488291 22071 leader_election.cc:302] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3806e18f82f1475fad4d83c4841514b9, e31ec108b1ea4b1e80f10cffde040ad1; no voters:
I20260808 03:00:42.488710 22680 raft_consensus.cc:2781] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:00:42.488865 22680 raft_consensus.cc:482] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:00:42.488976 22680 raft_consensus.cc:3037] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:42.491650 22680 raft_consensus.cc:504] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:42.492235 22680 leader_election.cc:288] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [CANDIDATE]: Term 1 election: Requested vote from peers 181eb68089d94205961cde087fb6cc09 (127.0.205.130:34615), e31ec108b1ea4b1e80f10cffde040ad1 (127.0.205.131:41689)
I20260808 03:00:42.492684 22365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c4177045644179926f6d0173bdf44d" candidate_uuid: "3806e18f82f1475fad4d83c4841514b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181eb68089d94205961cde087fb6cc09"
I20260808 03:00:42.492692 22569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1c4177045644179926f6d0173bdf44d" candidate_uuid: "3806e18f82f1475fad4d83c4841514b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e31ec108b1ea4b1e80f10cffde040ad1"
I20260808 03:00:42.492884 22365 raft_consensus.cc:3037] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:42.492913 22569 raft_consensus.cc:3037] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:00:42.495266 22569 raft_consensus.cc:2454] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3806e18f82f1475fad4d83c4841514b9 in term 1.
I20260808 03:00:42.495272 22365 raft_consensus.cc:2454] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3806e18f82f1475fad4d83c4841514b9 in term 1.
I20260808 03:00:42.495734 22072 leader_election.cc:302] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181eb68089d94205961cde087fb6cc09, 3806e18f82f1475fad4d83c4841514b9; no voters:
I20260808 03:00:42.496095 22680 raft_consensus.cc:2781] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:00:42.496562 22680 raft_consensus.cc:686] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 1 LEADER]: Becoming Leader. State: Replica: 3806e18f82f1475fad4d83c4841514b9, State: Running, Role: LEADER
I20260808 03:00:42.497043 22680 consensus_queue.cc:237] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } } peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } }
I20260808 03:00:42.502135 21948 catalog_manager.cc:5495] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3806e18f82f1475fad4d83c4841514b9 (127.0.205.129). New cstate: current_term: 1 leader_uuid: "3806e18f82f1475fad4d83c4841514b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806e18f82f1475fad4d83c4841514b9" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 43121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 } health_report { overall_health: UNKNOWN } } }
I20260808 03:00:42.523275 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.517726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49730 (local address 127.0.205.129:43121)
0808 03:00:42.517991 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.517995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.518013 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:00:42.518185 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.518189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.518270 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.518408 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:00:42.518757 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.519483 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.521038 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.521483 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.521588 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.522174 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:00:42.522721 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:00:42.522843 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.522932 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.523002 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 03:00:42.532266 22365 raft_consensus.cc:1261] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09 [term 1 FOLLOWER]: Refusing update from remote peer 3806e18f82f1475fad4d83c4841514b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:00:42.532265 22569 raft_consensus.cc:1261] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1 [term 1 FOLLOWER]: Refusing update from remote peer 3806e18f82f1475fad4d83c4841514b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:00:42.533505 22685 consensus_queue.cc:1035] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e31ec108b1ea4b1e80f10cffde040ad1" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 41689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:00:42.533919 22680 consensus_queue.cc:1035] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "181eb68089d94205961cde087fb6cc09" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:00:42.550034 22688 mvcc.cc:205] Tried to move back new op lower bound from 7316103342199951360 to 7316103342071787520. Current Snapshot: MvccSnapshot[applied={T|T < 7316103342199951360}]
I20260808 03:00:42.562731 21948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:00:42.563053 21948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:00:42.567085 21948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Sending DeleteTablet for 3 replicas of tablet f1c4177045644179926f6d0173bdf44d
I20260808 03:00:42.568405 22337 tablet_service.cc:1514] Processing DeleteTablet for tablet f1c4177045644179926f6d0173bdf44d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:00:42 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:40882
I20260808 03:00:42.568567 22540 tablet_service.cc:1514] Processing DeleteTablet for tablet f1c4177045644179926f6d0173bdf44d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:00:42 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:34566
I20260808 03:00:42.568949 22142 tablet_service.cc:1514] Processing DeleteTablet for tablet f1c4177045644179926f6d0173bdf44d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:00:42 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:49718
I20260808 03:00:42.570425 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 22009
I20260808 03:00:42.578402 22702 tablet_replica.cc:330] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: stopping tablet replica
I20260808 03:00:42.578619 22701 tablet_replica.cc:330] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: stopping tablet replica
I20260808 03:00:42.578922 22702 raft_consensus.cc:2229] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:00:42.578981 22701 raft_consensus.cc:2229] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:00:42.579517 22701 raft_consensus.cc:2258] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:00:42.579695 22702 raft_consensus.cc:2258] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:00:42.580351 22700 tablet_replica.cc:330] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: stopping tablet replica
I20260808 03:00:42.580775 22700 raft_consensus.cc:2229] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:00:42.582568 22702 ts_tablet_manager.cc:1905] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:00:42.583035 22701 ts_tablet_manager.cc:1905] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:00:42.587610 22700 raft_consensus.cc:2258] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:00:42.590250 22700 ts_tablet_manager.cc:1905] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:00:42.591318 22701 ts_tablet_manager.cc:1918] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:00:42.591491 22701 log.cc:1196] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/f1c4177045644179926f6d0173bdf44d
I20260808 03:00:42.592067 22701 ts_tablet_manager.cc:1939] T f1c4177045644179926f6d0173bdf44d P e31ec108b1ea4b1e80f10cffde040ad1: Deleting consensus metadata
I20260808 03:00:42.594059 21931 catalog_manager.cc:4841] TS e31ec108b1ea4b1e80f10cffde040ad1 (127.0.205.131:41689): tablet f1c4177045644179926f6d0173bdf44d (table test-table [id=6a79b225c41d4573944b16fab28d94f7]) successfully deleted
I20260808 03:00:42.594308 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.571943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49736 (local address 127.0.205.129:43121)
0808 03:00:42.572170 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.572179 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.572203 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:00:42.572286 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.572294 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.572404 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.572540 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:42.574970 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.576775 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.579445 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.580177 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.580366 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.584364 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:42.584386 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:42.584405 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:42.584465 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:42.589343 (+ 4878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:42.590480 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:42.590489 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:42.590494 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:42.590732 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:42.591201 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:42.591206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:42.591210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:42.591844 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:42.592243 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.593525 (+ 1282us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.593778 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 03:00:42.595357 22218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:42.598105 22700 ts_tablet_manager.cc:1918] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:00:42.598285 22700 log.cc:1196] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/f1c4177045644179926f6d0173bdf44d
I20260808 03:00:42.598857 22700 ts_tablet_manager.cc:1939] T f1c4177045644179926f6d0173bdf44d P 181eb68089d94205961cde087fb6cc09: Deleting consensus metadata
I20260808 03:00:42.600772 21932 catalog_manager.cc:4841] TS 181eb68089d94205961cde087fb6cc09 (127.0.205.130:34615): tablet f1c4177045644179926f6d0173bdf44d (table test-table [id=6a79b225c41d4573944b16fab28d94f7]) successfully deleted
I20260808 03:00:42.713155 22702 ts_tablet_manager.cc:1918] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:00:42.713466 22702 log.cc:1196] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/f1c4177045644179926f6d0173bdf44d
I20260808 03:00:42.713960 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 22009
W20260808 03:00:42.740687 21929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.205.129:43121 (error 108)
I20260808 03:00:42.740995 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 22232
I20260808 03:00:42.742825 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.742309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43121 (local address 127.0.0.1:49742)
0808 03:00:42.742532 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.742673 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260808 03:00:42.743086 21929 catalog_manager.cc:4584] TS 3806e18f82f1475fad4d83c4841514b9 (127.0.205.129:43121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1c4177045644179926f6d0173bdf44d: Network error: Client connection negotiation failed: client connection to 127.0.205.129:43121: connect: Connection refused (error 111)
I20260808 03:00:42.756026 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.741816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40896 (local address 127.0.205.130:34615)
0808 03:00:42.742085 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.742094 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.742200 (+ 106us) server_negotiation.cc:407] Connection header received
0808 03:00:42.742436 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.742442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.742541 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.742664 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:42.744245 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.744992 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.746611 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.747022 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.747157 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.750015 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:42.750030 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:42.750046 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:42.750081 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:42.752948 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:42.753746 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:42.753755 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:42.753758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:42.753949 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:42.754354 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:42.754360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:42.754364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:42.754779 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:42.755118 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.755410 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.755658 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260808 03:00:42.756732 22429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:42.797142 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.796594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43121 (local address 127.0.0.1:49754)
0808 03:00:42.796837 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.796967 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 03:00:42.798483 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.797987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43121 (local address 127.0.0.1:49762)
0808 03:00:42.798173 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.798340 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 03:00:42.847757 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 22232
I20260808 03:00:42.858946 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.858445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43121 (local address 127.0.0.1:49778)
0808 03:00:42.858666 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.858766 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 03:00:42.860071 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.859727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43121 (local address 127.0.0.1:49792)
0808 03:00:42.859882 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.859964 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 03:00:42.875260 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 22444
I20260808 03:00:42.889349 22668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.876068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34574 (local address 127.0.205.131:41689)
0808 03:00:42.876237 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 03:00:42.876242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:00:42.876261 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:00:42.876446 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:00:42.876451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:00:42.876521 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:00:42.876611 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:00:42.878077 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.878835 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.880127 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:00:42.880520 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:00:42.880612 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:00:42.883695 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:00:42.883711 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:00:42.883729 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:00:42.883761 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:00:42.886736 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:42.887405 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:42.887412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:42.887414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:42.887553 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:00:42.887903 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:00:42.887906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:00:42.887908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:00:42.888268 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:00:42.888515 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 03:00:42.888837 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 03:00:42.889076 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:00:42.890089 22636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:42.941454 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.940945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43121 (local address 127.0.0.1:49804)
0808 03:00:42.941188 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.941285 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 03:00:42.942713 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:42.942288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:43121 (local address 127.0.0.1:49820)
0808 03:00:42.942500 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:42.942586 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260808 03:00:42.983530 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 22444
I20260808 03:00:43.007009 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 21907
I20260808 03:00:43.008459 21997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:00:43.126438 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 21907
I20260808 03:00:43.156617 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:35547
--webserver_interface=127.0.205.190
--webserver_port=41229
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:35547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:43.525569 22713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:43.526039 22713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:43.526230 22713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:43.533939 22713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:00:43.534124 22713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:43.534227 22713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:43.534339 22713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:00:43.534431 22713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:00:43.545284 22713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:35547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:35547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=41229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:43.546864 22713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:43.548276 22713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:43.561378 22725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:43.561283 22730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:43.561417 22727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:43.562587 22713 server_base.cc:1034] running on GCE node
I20260808 03:00:43.563506 22713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:43.565062 22713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:43.566306 22713 hybrid_clock.cc:648] HybridClock initialized: now 1786158043566238 us; error 56 us; skew 500 ppm
I20260808 03:00:43.566675 22713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:43 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158043, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:43.570093 22713 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:00:43.571593 22713 webserver.cc:460] Webserver started at http://127.0.205.190:41229/ using document root <none> and password file <none>
I20260808 03:00:43.572290 22713 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:43.572405 22713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:43.578044 22713 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 03:00:43.581416 22743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:43.583029 22713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:00:43.583243 22713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "fa049867d0664e22838a4d12f7e50dc3"
format_stamp: "Formatted at 2026-08-08 03:00:37 on dist-test-slave-6ntk"
server_key: "71bb3a64ef4468bb335de202596cf082"
server_key_iv: "bc7ca34db324690da1ab79a970908704"
server_key_version: "encryptionkey@0"
I20260808 03:00:43.584344 22713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:43.601298 22713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:43.606523 22713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:43.606868 22713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:43.626796 22713 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:35547
I20260808 03:00:43.626817 22822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:35547 every 8 connection(s)
I20260808 03:00:43.628486 22713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:00:43.635958 22823 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:00:43.636794 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 22713
I20260808 03:00:43.637681 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:43121
--local_ip_for_outbound_sockets=127.0.205.129
--tserver_master_addrs=127.0.205.190:35547
--webserver_port=44825
--webserver_interface=127.0.205.129
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:00:43.639672 22823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Bootstrap starting.
I20260808 03:00:43.660650 22823 log.cc:824] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Log is configured to *not* fsync() on all Append() calls
I20260808 03:00:43.677568 22823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:00:43.678236 22823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3: Bootstrap complete.
I20260808 03:00:43.687727 22823 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:43.688896 22823 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa049867d0664e22838a4d12f7e50dc3, State: Initialized, Role: FOLLOWER
I20260808 03:00:43.689603 22823 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:43.689913 22823 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:00:43.690042 22823 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:00:43.690250 22823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 2 FOLLOWER]: Advancing to term 3
I20260808 03:00:43.694351 22823 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:43.694967 22823 leader_election.cc:302] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa049867d0664e22838a4d12f7e50dc3; no voters:
I20260808 03:00:43.695736 22823 leader_election.cc:288] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 03:00:43.695878 22829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 03:00:43.697400 22829 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [term 3 LEADER]: Becoming Leader. State: Replica: fa049867d0664e22838a4d12f7e50dc3, State: Running, Role: LEADER
I20260808 03:00:43.698137 22829 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } }
I20260808 03:00:43.698470 22823 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:00:43.700855 22831 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa049867d0664e22838a4d12f7e50dc3. Latest consensus state: current_term: 3 leader_uuid: "fa049867d0664e22838a4d12f7e50dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } } }
I20260808 03:00:43.701340 22831 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:43.701233 22830 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fa049867d0664e22838a4d12f7e50dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa049867d0664e22838a4d12f7e50dc3" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 35547 } } }
I20260808 03:00:43.701577 22830 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa049867d0664e22838a4d12f7e50dc3 [sys.catalog]: This master's current role is: LEADER
I20260808 03:00:43.704550 22843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:00:43.713989 22843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:00:43.715660 22843 catalog_manager.cc:1259] Loaded cluster ID: edd28047407044da82538fd1ebb1829c
I20260808 03:00:43.715847 22843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 03:00:43.718137 22843 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0738a199ad google::LogMessage::Fail() at ??:0
@ 0x7f0738a1c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f0738a194de google::LogMessage::Flush() at ??:0
@ 0x7f0738a1aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f074e44c512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f074e449976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f074e4d1610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f074e4d1439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f073b665345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f073b66891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f073b668799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f073b643eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f073e91d6db start_thread at ??:0
@ 0x7f07378b361f clone at ??:0
W20260808 03:00:44.066131 22826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:44.066432 22826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:44.066636 22826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:44.074242 22826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:44.074369 22826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:44.074549 22826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:00:44.084714 22826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:43121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=44825
--tserver_master_addrs=127.0.205.190:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:44.086206 22826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:44.087615 22826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:44.101996 22897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:44.101971 22893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:44.102468 22889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:44.102687 22826 server_base.cc:1034] running on GCE node
I20260808 03:00:44.103488 22826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:44.104578 22826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:44.105821 22826 hybrid_clock.cc:648] HybridClock initialized: now 1786158044105772 us; error 44 us; skew 500 ppm
I20260808 03:00:44.106204 22826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:44 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:44.109812 22826 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:00:44.111726 22826 webserver.cc:460] Webserver started at http://127.0.205.129:44825/ using document root <none> and password file <none>
I20260808 03:00:44.112557 22826 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:44.112720 22826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:44.118813 22826 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:00:44.121989 22906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:44.123495 22826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:00:44.123692 22826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "3806e18f82f1475fad4d83c4841514b9"
format_stamp: "Formatted at 2026-08-08 03:00:38 on dist-test-slave-6ntk"
server_key: "9d4af750b95b416e3184e2bc7782ef6d"
server_key_iv: "182fe700636d36d258c81357c047678b"
server_key_version: "encryptionkey@0"
I20260808 03:00:44.124770 22826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:44.148231 22826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:44.152992 22826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:44.153293 22826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:44.154417 22826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:44.156291 22915 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260808 03:00:44.161855 22915 ts_tablet_manager.cc:1852] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260808 03:00:44.162907 22915 ts_tablet_manager.cc:1905] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:00:44.170521 22915 ts_tablet_manager.cc:1918] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:00:44.170773 22915 log.cc:1196] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/f1c4177045644179926f6d0173bdf44d
I20260808 03:00:44.171267 22915 ts_tablet_manager.cc:1939] T f1c4177045644179926f6d0173bdf44d P 3806e18f82f1475fad4d83c4841514b9: Deleting consensus metadata
I20260808 03:00:44.171727 22826 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260808 03:00:44.171887 22826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260808 03:00:44.172043 22826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:44.172154 22826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:44.212402 22826 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:43121
I20260808 03:00:44.212455 23066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:43121 every 8 connection(s)
I20260808 03:00:44.213804 22826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:00:44.219066 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:44.217128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:49673)
0808 03:00:44.217935 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:44.218181 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":292,"threads_started":1}
I20260808 03:00:44.219362 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 22826
W20260808 03:00:44.219722 23068 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:35547 (0 consecutive failures): Network error: Failed to ping master at 127.0.205.190:35547: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
I20260808 03:00:44.220111 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:34615
--local_ip_for_outbound_sockets=127.0.205.130
--tserver_master_addrs=127.0.205.190:35547
--webserver_port=44969
--webserver_interface=127.0.205.130
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:00:44.220745 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:44.220283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:50793)
0808 03:00:44.220497 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:44.220594 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 03:00:44.221827 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:44.221409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:57963)
0808 03:00:44.221546 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:44.221663 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 03:00:44.222141 23068 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 03:00:44.579720 23072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:44.580071 23072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:44.580173 23072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:44.587580 23072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:44.587711 23072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:44.587842 23072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:00:44.598016 23072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:34615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=44969
--tserver_master_addrs=127.0.205.190:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:44.599486 23072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:44.601250 23072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:44.615476 23084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:44.615464 23085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:44.615497 23087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:44.616827 23072 server_base.cc:1034] running on GCE node
I20260808 03:00:44.617558 23072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:44.618746 23072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:44.620011 23072 hybrid_clock.cc:648] HybridClock initialized: now 1786158044619941 us; error 69 us; skew 500 ppm
I20260808 03:00:44.620375 23072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:44 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:44.624161 23072 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:00:44.625756 23072 webserver.cc:460] Webserver started at http://127.0.205.130:44969/ using document root <none> and password file <none>
I20260808 03:00:44.626577 23072 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:44.626693 23072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:44.632651 23072 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 03:00:44.635578 23095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:44.637077 23072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:00:44.637231 23072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "181eb68089d94205961cde087fb6cc09"
format_stamp: "Formatted at 2026-08-08 03:00:38 on dist-test-slave-6ntk"
server_key: "d6bd487298ab60f77181d3d3e13c8e3d"
server_key_iv: "60dbf4ad6b9732844eed2f16463bad52"
server_key_version: "encryptionkey@0"
I20260808 03:00:44.638301 23072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:44.656447 23072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:44.661032 23072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:44.661298 23072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:44.662364 23072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:44.663697 23072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:44.663798 23072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:44.663879 23072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:44.663936 23072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:44.699494 23072 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:34615
I20260808 03:00:44.699642 23264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:34615 every 8 connection(s)
I20260808 03:00:44.701073 23072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:00:44.706513 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:44.704404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:41865)
0808 03:00:44.705204 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:44.705479 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":311,"threads_started":1}
W20260808 03:00:44.707221 23265 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:35547 (0 consecutive failures): Network error: Failed to ping master at 127.0.205.190:35547: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
I20260808 03:00:44.708230 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:44.707865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:41389)
0808 03:00:44.708038 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:44.708111 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 03:00:44.709177 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:44.708806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:40893)
0808 03:00:44.708965 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:44.709057 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 03:00:44.709452 23265 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 03:00:44.710747 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 23072
I20260808 03:00:44.711444 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:41689
--local_ip_for_outbound_sockets=127.0.205.131
--tserver_master_addrs=127.0.205.190:35547
--webserver_port=44401
--webserver_interface=127.0.205.131
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:00:45.062076 23273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:00:45.062376 23273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:00:45.062541 23273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:00:45.069928 23273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:00:45.070083 23273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:00:45.070287 23273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:00:45.081024 23273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:41689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=44401
--tserver_master_addrs=127.0.205.190:35547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:00:45.082536 23273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:00:45.083904 23273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:00:45.098052 23279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:45.098008 23284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:00:45.098098 23280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:00:45.099434 23273 server_base.cc:1034] running on GCE node
I20260808 03:00:45.100137 23273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:00:45.101230 23273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:00:45.102473 23273 hybrid_clock.cc:648] HybridClock initialized: now 1786158045102400 us; error 71 us; skew 500 ppm
I20260808 03:00:45.102819 23273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:00:45 dist-test-slave-6ntk krb5kdc[21062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158045, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:00:45.106290 23273 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:00:45.108043 23273 webserver.cc:460] Webserver started at http://127.0.205.131:44401/ using document root <none> and password file <none>
I20260808 03:00:45.108758 23273 fs_manager.cc:359] Metadata directory not provided
I20260808 03:00:45.108873 23273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:00:45.114802 23273 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 03:00:45.117519 23293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:45.119141 23273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:00:45.119287 23273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "e31ec108b1ea4b1e80f10cffde040ad1"
format_stamp: "Formatted at 2026-08-08 03:00:39 on dist-test-slave-6ntk"
server_key: "f63ddda69036083232efa6a92d538238"
server_key_iv: "6a158b3dfe0da0ce5bd28eabedecc561"
server_key_version: "encryptionkey@0"
I20260808 03:00:45.120252 23273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:00:45.148521 23273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:00:45.153198 23273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:00:45.153450 23273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:00:45.154516 23273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:00:45.155839 23273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:00:45.155929 23273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:45.156002 23273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:00:45.156061 23273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:00:45.189774 23273 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:41689
I20260808 03:00:45.189865 23464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:41689 every 8 connection(s)
I20260808 03:00:45.191395 23273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:00:45.191982 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 23273
I20260808 03:00:45.196723 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:45.194870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:58853)
0808 03:00:45.195593 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:45.195851 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":278,"threads_started":1}
W20260808 03:00:45.197557 23465 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:35547 (0 consecutive failures): Network error: Failed to ping master at 127.0.205.190:35547: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
I20260808 03:00:45.198395 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:45.198076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:37153)
0808 03:00:45.198209 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:45.198281 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 03:00:45.199198 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:45.198895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:58629)
0808 03:00:45.199019 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:45.199091 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 03:00:45.199436 23465 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 03:00:45.223702 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:45.222801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:57609)
0808 03:00:45.223293 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:45.223409 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":221,"threads_started":1}
I20260808 03:00:45.710983 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:45.710114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:54281)
0808 03:00:45.710589 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:45.710699 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":230,"threads_started":1}
I20260808 03:00:46.200886 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:46.200044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:40563)
0808 03:00:46.200534 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:46.200641 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":246,"threads_started":1}
I20260808 03:00:46.225644 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:46.224684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:39415)
0808 03:00:46.225288 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:46.225446 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":338,"threads_started":1}
I20260808 03:00:46.713037 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:46.712099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:43749)
0808 03:00:46.712663 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:46.712780 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":251,"threads_started":1}
I20260808 03:00:47.202787 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:47.201911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:43647)
0808 03:00:47.202488 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:47.202596 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":309,"threads_started":1}
I20260808 03:00:47.227447 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:47.226669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:37703)
0808 03:00:47.227157 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:47.227255 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":244,"threads_started":1}
I20260808 03:00:47.715010 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:47.714130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:50767)
0808 03:00:47.714722 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:47.714817 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":291,"threads_started":1}
I20260808 03:00:48.204433 23482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:48.203699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:35121)
0808 03:00:48.204147 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:48.204236 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":239,"threads_started":1}
I20260808 03:00:48.229621 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:48.228623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:59293)
0808 03:00:48.229291 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:48.229412 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":359,"threads_started":1}
I20260808 03:00:48.717011 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:48.716123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:35111)
0808 03:00:48.716682 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:48.716782 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":278,"threads_started":1}
I20260808 03:00:49.206494 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:49.205533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:56559)
0808 03:00:49.206150 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:49.206266 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":273,"threads_started":1}
I20260808 03:00:49.231345 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:49.230631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:58417)
0808 03:00:49.231079 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:49.231165 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":246,"threads_started":1}
I20260808 03:00:49.719035 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:49.718190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:33055)
0808 03:00:49.718735 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:49.718829 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":283,"threads_started":1}
I20260808 03:00:50.208671 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:50.207769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:35531)
0808 03:00:50.208404 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:50.208491 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":344,"threads_started":1}
I20260808 03:00:50.232928 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:50.232221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:35329)
0808 03:00:50.232665 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:50.232757 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":229,"threads_started":1}
I20260808 03:00:50.721132 23492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:50.720121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:51983)
0808 03:00:50.720805 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:50.720956 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":310,"threads_started":1}
I20260808 03:00:51.210485 23493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:51.209674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:51685)
0808 03:00:51.210209 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:51.210306 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":309,"threads_started":1}
I20260808 03:00:51.234764 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:51.233861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:35003)
0808 03:00:51.234456 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:51.234549 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":231,"threads_started":1}
I20260808 03:00:51.723125 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:51.722220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:36413)
0808 03:00:51.722815 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:51.722902 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":253,"threads_started":1}
I20260808 03:00:52.212483 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:52.211625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:50895)
0808 03:00:52.212159 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:52.212255 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":265,"threads_started":1}
I20260808 03:00:52.236575 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:52.235788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:58633)
0808 03:00:52.236277 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:52.236370 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":260,"threads_started":1}
I20260808 03:00:52.725502 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:52.724360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:37953)
0808 03:00:52.725024 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:52.725157 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":306,"threads_started":1}
I20260808 03:00:53.215356 23501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:53.213717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:54967)
0808 03:00:53.215110 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:53.215197 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1116,"thread_start_us":957,"threads_started":1}
I20260808 03:00:53.238499 23502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:53.237579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:42577)
0808 03:00:53.238217 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:53.238309 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":335,"threads_started":1}
I20260808 03:00:53.727319 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:53.726573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:53611)
0808 03:00:53.727060 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:53.727142 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":236,"threads_started":1}
I20260808 03:00:54.217280 23505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:54.216339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:59409)
0808 03:00:54.216920 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:54.217051 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":295,"threads_started":1}
I20260808 03:00:54.240302 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:54.239539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:57343)
0808 03:00:54.240015 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:54.240111 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":261,"threads_started":1}
I20260808 03:00:54.729130 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:54.728312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:44373)
0808 03:00:54.728871 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:54.728961 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":328,"threads_started":1}
I20260808 03:00:55.219458 23508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:55.218569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:35051)
0808 03:00:55.219148 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:55.219256 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":306,"threads_started":1}
I20260808 03:00:55.242424 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:55.241408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:57089)
0808 03:00:55.242133 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:55.242217 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":400,"threads_started":1}
I20260808 03:00:55.731006 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:55.730199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:47555)
0808 03:00:55.730717 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:55.730810 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":272,"threads_started":1}
I20260808 03:00:56.221325 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:56.220465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:36113)
0808 03:00:56.220972 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:56.221128 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":267,"threads_started":1}
I20260808 03:00:56.244089 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:56.243358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:57041)
0808 03:00:56.243813 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:56.243921 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":238,"threads_started":1}
I20260808 03:00:56.732887 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:56.732107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:39213)
0808 03:00:56.732615 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:56.732708 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":270,"threads_started":1}
I20260808 03:00:57.223088 23521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:57.222270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:58323)
0808 03:00:57.222801 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:57.222930 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":243,"threads_started":1}
I20260808 03:00:57.245811 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:57.245027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:54713)
0808 03:00:57.245505 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:57.245616 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":243,"threads_started":1}
I20260808 03:00:57.734823 23525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:57.733941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:47251)
0808 03:00:57.734501 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:57.734611 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":262,"threads_started":1}
I20260808 03:00:58.224808 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:58.224029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:40881)
0808 03:00:58.224488 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:58.224588 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":223,"threads_started":1}
I20260808 03:00:58.247674 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:58.246813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:36553)
0808 03:00:58.247369 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:58.247469 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":278,"threads_started":1}
I20260808 03:00:58.736598 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:58.735800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:42923)
0808 03:00:58.736288 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:58.736383 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":231,"threads_started":1}
I20260808 03:00:59.226835 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:59.225832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:50435)
0808 03:00:59.226451 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:59.226550 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":266,"threads_started":1}
I20260808 03:00:59.250007 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:59.249065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:46253)
0808 03:00:59.249616 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:59.249728 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":297,"threads_started":1}
I20260808 03:00:59.738763 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:00:59.738000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.130:36457)
0808 03:00:59.738468 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 03:00:59.738565 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":238,"threads_started":1}
I20260808 03:01:00.197616 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 22826
I20260808 03:01:00.221622 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:00.199223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.0.205.129:43121)
0808 03:01:00.199855 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:00.199870 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:00.199911 (+ 41us) server_negotiation.cc:407] Connection header received
0808 03:01:00.201621 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:00.201633 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:00.202167 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:00.202490 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:00.203944 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:00.205184 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:00.206748 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:00.207448 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:00.207594 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:00.214153 (+ 6559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:00.214169 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:00.214192 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:00.214245 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:00.218506 (+ 4261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:00.219255 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:00.219261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:00.219269 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:00.219445 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:00.219778 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:00.219781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:00.219783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:00.220347 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:00.220586 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:00.220961 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 03:01:00.221316 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":339,"threads_started":1}
I20260808 03:01:00.223134 23061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:00.228951 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:00.228036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.131:42077)
0808 03:01:00.228562 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:00.228705 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":236,"threads_started":1}
I20260808 03:01:00.312418 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:00.311332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:35547 (local address 127.0.205.129:45863)
0808 03:01:00.311983 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:00.312156 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:35547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":313,"threads_started":1}
I20260808 03:01:00.312419 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 22826
I20260808 03:01:00.340292 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 23072
I20260808 03:01:00.359481 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:00.341378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.0.205.130:34615)
0808 03:01:00.342022 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:00.342037 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:00.342074 (+ 37us) server_negotiation.cc:407] Connection header received
0808 03:01:00.343757 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:00.343766 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:00.344212 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:00.344573 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:00.346237 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:00.347470 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:00.349287 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:00.350041 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:00.350247 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:00.353051 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:00.353059 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:00.353074 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:00.353114 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:00.356203 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:00.356858 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:00.356865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:00.356871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:00.357054 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:00.357417 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:00.357420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:00.357422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:00.358025 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:00.358273 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:00.358733 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 03:01:00.359181 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":280,"threads_started":1}
I20260808 03:01:00.360934 23257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:00.445487 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 23072
I20260808 03:01:00.465857 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 23273
I20260808 03:01:00.484220 23543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:00.466944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37268 (local address 127.0.205.131:41689)
0808 03:01:00.467570 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:00.467585 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:00.467628 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:01:00.469318 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:00.469337 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:00.469772 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:00.470192 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:00.471755 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:00.472990 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:00.474341 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:00.475005 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:00.475213 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:00.478001 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:00.478010 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:00.478025 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:00.478070 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:00.480872 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:00.481557 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:00.481571 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:00.481578 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:00.481748 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:00.482164 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:00.482170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:00.482173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:00.482709 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:00.483030 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:00.483497 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 03:01:00.483907 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":278,"threads_started":1}
I20260808 03:01:00.485682 23460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:00.570412 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 23273
2026-08-08T03:01:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25205 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:01:00 dist-test-slave-6ntk krb5kdc[23547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:01:00 dist-test-slave-6ntk krb5kdc[23547](info): set up 2 sockets
Aug 08 03:01:00 dist-test-slave-6ntk krb5kdc[23547](info): commencing operation
krb5kdc: starting...
W20260808 03:01:01.933569 822 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:01:02 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:01:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:01:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:01:02.250428 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:36925
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:36925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:01:02.614854 23571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:02.615176 23571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:02.615353 23571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:02.622862 23571 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 03:01:02.622977 23571 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 03:01:02.623066 23571 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 03:01:02.623164 23571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:01:02.623270 23571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:02.623385 23571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:02.623473 23571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:01:02.623554 23571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:01:02.634114 23571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:36925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:36925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:02.635596 23571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:02.637017 23571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:02.650362 23577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:02.650424 23571 server_base.cc:1034] running on GCE node
W20260808 03:01:02.650736 23580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:02.650537 23578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:02.651827 23571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:02.653241 23571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:02.654472 23571 hybrid_clock.cc:648] HybridClock initialized: now 1786158062654402 us; error 34 us; skew 500 ppm
I20260808 03:01:02.654795 23571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:02 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158062, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:02.661679 23571 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:01:02.663285 23571 webserver.cc:460] Webserver started at http://127.0.205.190:35623/ using document root <none> and password file <none>
I20260808 03:01:02.664006 23571 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:02.664103 23571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:02.664379 23571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:01:02.667553 23571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "ada042949a114aef932790d2e07a68a1"
format_stamp: "Formatted at 2026-08-08 03:01:02 on dist-test-slave-6ntk"
server_key: "095c939b18ae905754a6f75a181a093d"
server_key_iv: "6ccb5f22f385859f5816602c385cf3f7"
server_key_version: "encryptionkey@0"
I20260808 03:01:02.668305 23571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "ada042949a114aef932790d2e07a68a1"
format_stamp: "Formatted at 2026-08-08 03:01:02 on dist-test-slave-6ntk"
server_key: "095c939b18ae905754a6f75a181a093d"
server_key_iv: "6ccb5f22f385859f5816602c385cf3f7"
server_key_version: "encryptionkey@0"
I20260808 03:01:02.673939 23571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 03:01:02.677851 23588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:02.679291 23571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 03:01:02.679454 23571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "ada042949a114aef932790d2e07a68a1"
format_stamp: "Formatted at 2026-08-08 03:01:02 on dist-test-slave-6ntk"
server_key: "095c939b18ae905754a6f75a181a093d"
server_key_iv: "6ccb5f22f385859f5816602c385cf3f7"
server_key_version: "encryptionkey@0"
I20260808 03:01:02.679646 23571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:02.694087 23571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:02.698454 23571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:02.698695 23571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:02.718281 23571 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:36925
I20260808 03:01:02.718303 23675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:36925 every 8 connection(s)
I20260808 03:01:02.719815 23571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:01:02.721396 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 23571
I20260808 03:01:02.721654 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:01:02.722530 822 external_mini_cluster.cc:1428] Setting key 2376b9b13284ba7d7e8cdd7032302317
I20260808 03:01:02.724845 23676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:01:02.736384 23676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Bootstrap starting.
Aug 08 03:01:02 dist-test-slave-6ntk krb5kdc[23547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:01:02.740090 23676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Neither blocks nor log segments found. Creating new log.
I20260808 03:01:02.741307 23676 log.cc:824] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Log is configured to *not* fsync() on all Append() calls
I20260808 03:01:02.746001 23676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: No bootstrap required, opened a new log
I20260808 03:01:02.747921 23680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:02.724767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57512 (local address 127.0.205.190:36925)
0808 03:01:02.725582 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:02.725595 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:02.725638 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:01:02.727373 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:02.727397 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:02.727796 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:02.728208 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:02.729766 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:02.731155 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:02.732529 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:02.733235 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:02.733460 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:02.737978 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:02.737990 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:02.738005 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:02.738056 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:02.741190 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:02.742318 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:02.742333 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:02.742350 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:02.743043 (+ 693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:02.743570 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:02.743575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:02.743578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:02.744499 (+ 921us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:02.744846 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:02.745641 (+ 795us) server_negotiation.cc:299] Negotiation successful
0808 03:01:02.746301 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":344,"threads_started":1}
I20260808 03:01:02.757843 23676 raft_consensus.cc:348] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:02.758435 23676 raft_consensus.cc:374] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:01:02.758569 23676 raft_consensus.cc:729] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada042949a114aef932790d2e07a68a1, State: Initialized, Role: FOLLOWER
I20260808 03:01:02.759384 23676 consensus_queue.cc:260] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:02.759759 23676 raft_consensus.cc:388] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:01:02.759963 23676 raft_consensus.cc:482] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:01:02.760211 23676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:01:02.764175 23676 raft_consensus.cc:504] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:02.765004 23676 leader_election.cc:302] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ada042949a114aef932790d2e07a68a1; no voters:
I20260808 03:01:02.765712 23676 leader_election.cc:288] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:01:02.765856 23682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:01:02.766626 23682 raft_consensus.cc:686] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 1 LEADER]: Becoming Leader. State: Replica: ada042949a114aef932790d2e07a68a1, State: Running, Role: LEADER
I20260808 03:01:02.767225 23682 consensus_queue.cc:237] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:02.768098 23676 sys_catalog.cc:564] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:01:02.770473 23684 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ada042949a114aef932790d2e07a68a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } } }
I20260808 03:01:02.770994 23684 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:01:02.772641 23695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:01:02.773759 23687 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ada042949a114aef932790d2e07a68a1. Latest consensus state: current_term: 1 leader_uuid: "ada042949a114aef932790d2e07a68a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } } }
I20260808 03:01:02.774158 23687 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:01:02.786141 23695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:01:02.803730 23695 catalog_manager.cc:1347] Generated new cluster ID: 68b418f9dfb043c2bb0de8afcf8b0a49
I20260808 03:01:02.803874 23695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:01:02.825800 23695 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:01:02.827194 23695 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:01:02.836902 23695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Generated new TSK 0
I20260808 03:01:02.837834 23695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 03:01:02.931098 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36925
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:01:03.325860 23709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:03.326284 23709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:03.326467 23709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:03.334061 23709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:03.334227 23709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:03.334436 23709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:01:03.344929 23709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:03.346473 23709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:03.347935 23709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:03.362411 23718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:03.362519 23722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:03.362622 23717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:03.367566 23709 server_base.cc:1034] running on GCE node
I20260808 03:01:03.368366 23709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:03.369549 23709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:03.370772 23709 hybrid_clock.cc:648] HybridClock initialized: now 1786158063370699 us; error 71 us; skew 500 ppm
I20260808 03:01:03.371130 23709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:03 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158063, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:03.374524 23709 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:01:03.376037 23709 webserver.cc:460] Webserver started at http://127.0.205.129:42981/ using document root <none> and password file <none>
I20260808 03:01:03.376798 23709 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:03.376951 23709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:03.377278 23709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:01:03.380362 23709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "efae2f315c024dabb8fa0bc555cd3147"
format_stamp: "Formatted at 2026-08-08 03:01:03 on dist-test-slave-6ntk"
server_key: "8c76dbe72c9198b0feff4d914e742061"
server_key_iv: "c63174d185cde5643b18801793baa992"
server_key_version: "encryptionkey@0"
I20260808 03:01:03.381178 23709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "efae2f315c024dabb8fa0bc555cd3147"
format_stamp: "Formatted at 2026-08-08 03:01:03 on dist-test-slave-6ntk"
server_key: "8c76dbe72c9198b0feff4d914e742061"
server_key_iv: "c63174d185cde5643b18801793baa992"
server_key_version: "encryptionkey@0"
I20260808 03:01:03.386927 23709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 03:01:03.390619 23732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:03.391975 23709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:01:03.392163 23709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "efae2f315c024dabb8fa0bc555cd3147"
format_stamp: "Formatted at 2026-08-08 03:01:03 on dist-test-slave-6ntk"
server_key: "8c76dbe72c9198b0feff4d914e742061"
server_key_iv: "c63174d185cde5643b18801793baa992"
server_key_version: "encryptionkey@0"
I20260808 03:01:03.392402 23709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:03.420037 23709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:03.424818 23709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:03.425150 23709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:03.426193 23709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:01:03.427671 23709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:01:03.427796 23709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:03.427944 23709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:01:03.428016 23709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:03.463879 23709 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:44389
I20260808 03:01:03.463933 23905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:44389 every 8 connection(s)
I20260808 03:01:03.465277 23709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:01:03.467629 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 23709
I20260808 03:01:03.467859 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 03:01:03.468814 822 external_mini_cluster.cc:1428] Setting key a65cf1cd06bbb29ad4d567bb645e0a4b
Aug 08 03:01:03 dist-test-slave-6ntk krb5kdc[23547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158063, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:01:03.488037 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:03.468339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:33497 (local address 127.0.205.190:36925)
0808 03:01:03.468895 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:03.468903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:03.470138 (+ 1235us) server_negotiation.cc:407] Connection header received
0808 03:01:03.472075 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:03.472084 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:03.472196 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:03.472317 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:03.473797 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:03.475033 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:03.476627 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:03.477345 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:03.477499 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:03.481242 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:03.481251 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:03.481258 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:03.481307 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:03.484549 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:03.485542 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:03.485546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:03.485549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:03.485670 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:03.486207 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:03.486210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:03.486212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:03.486478 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:03.486593 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:03.487573 (+ 980us) server_negotiation.cc:299] Negotiation successful
0808 03:01:03.487758 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":251,"threads_started":1}
I20260808 03:01:03.489612 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:03.469034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:33497)
0808 03:01:03.469974 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:03.470022 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:03.471779 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:03.472495 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:03.472503 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:03.473048 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:01:03.473620 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:03.473638 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:03.475193 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:03.475197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:03.476450 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:03.476468 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:03.477562 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:03.477566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:03.477771 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:03.478546 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:01:03.478569 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:01:03.480998 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:01:03.484702 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:03.484714 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:03.484735 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:03.485422 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:03.485783 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:03.485791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:03.485797 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:03.486092 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:03.486621 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:01:03.486634 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:01:03.487117 (+ 483us) client_negotiation.cc:769] Sending connection context
0808 03:01:03.487442 (+ 325us) client_negotiation.cc:240] Negotiation successful
0808 03:01:03.488055 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":386,"threads_started":1}
I20260808 03:01:03.491464 23909 heartbeater.cc:347] Connected to a master server at 127.0.205.190:36925
I20260808 03:01:03.491851 23909 heartbeater.cc:464] Registering TS with master...
I20260808 03:01:03.492669 23909 heartbeater.cc:511] Master 127.0.205.190:36925 requested a full tablet report, sending...
I20260808 03:01:03.494913 23616 ts_manager.cc:194] Registered new tserver with Master: efae2f315c024dabb8fa0bc555cd3147 (127.0.205.129:44389)
I20260808 03:01:03.496533 23616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:33497
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 03:01:03.538023 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36925
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:01:03.897094 23915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:03.897469 23915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:03.897584 23915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:03.905119 23915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:03.905301 23915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:03.905504 23915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:01:03.916011 23915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:03.917580 23915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:03.919003 23915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:03.932586 23921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:03.932633 23922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:03.932549 23928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:03.933382 23915 server_base.cc:1034] running on GCE node
I20260808 03:01:03.934083 23915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:03.935036 23915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:03.936254 23915 hybrid_clock.cc:648] HybridClock initialized: now 1786158063936198 us; error 37 us; skew 500 ppm
I20260808 03:01:03.936632 23915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:03 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158063, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:03.940297 23915 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:01:03.942023 23915 webserver.cc:460] Webserver started at http://127.0.205.130:44889/ using document root <none> and password file <none>
I20260808 03:01:03.942762 23915 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:03.942922 23915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:03.943253 23915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:01:03.946434 23915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "f9a6ae68adf04408b6b6528648c83295"
format_stamp: "Formatted at 2026-08-08 03:01:03 on dist-test-slave-6ntk"
server_key: "077baba89221530f917432ccca35017d"
server_key_iv: "8eedfe2c5156cd9ecaa0785ef300b16b"
server_key_version: "encryptionkey@0"
I20260808 03:01:03.947274 23915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "f9a6ae68adf04408b6b6528648c83295"
format_stamp: "Formatted at 2026-08-08 03:01:03 on dist-test-slave-6ntk"
server_key: "077baba89221530f917432ccca35017d"
server_key_iv: "8eedfe2c5156cd9ecaa0785ef300b16b"
server_key_version: "encryptionkey@0"
I20260808 03:01:03.952652 23915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:01:03.959756 23948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:03.962040 23915 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260808 03:01:03.962237 23915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "f9a6ae68adf04408b6b6528648c83295"
format_stamp: "Formatted at 2026-08-08 03:01:03 on dist-test-slave-6ntk"
server_key: "077baba89221530f917432ccca35017d"
server_key_iv: "8eedfe2c5156cd9ecaa0785ef300b16b"
server_key_version: "encryptionkey@0"
I20260808 03:01:03.962517 23915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:03.981614 23915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:03.987942 23915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:03.988261 23915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:03.989409 23915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:01:03.991148 23915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:01:03.991253 23915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:03.991405 23915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:01:03.991477 23915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:04.030550 23915 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:36127
I20260808 03:01:04.030561 24117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:36127 every 8 connection(s)
I20260808 03:01:04.031949 23915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:01:04.039127 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 23915
I20260808 03:01:04.039386 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 03:01:04.040107 822 external_mini_cluster.cc:1428] Setting key 2d518182b80b7925bb5e18e6e01f2b57
Aug 08 03:01:04 dist-test-slave-6ntk krb5kdc[23547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158063, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:01:04.059119 24123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.034994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:53707 (local address 127.0.205.190:36925)
0808 03:01:04.035473 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.035478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.036891 (+ 1413us) server_negotiation.cc:407] Connection header received
0808 03:01:04.038971 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.038977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.039078 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.039203 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:04.040848 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.041790 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.043937 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.044775 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.044964 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.050592 (+ 5628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:04.050601 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:04.050609 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:04.050658 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:04.054715 (+ 4057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:04.055863 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:04.055870 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:04.055876 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:04.056048 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:04.056676 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:04.056685 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:04.056691 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:04.057096 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:04.057256 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.058333 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.058612 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":251,"threads_started":1}
I20260808 03:01:04.060679 24124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.035775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:53707)
0808 03:01:04.036720 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:04.036770 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:04.038633 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:04.039475 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:04.039484 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:04.040063 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:01:04.040661 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.040678 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.041988 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.041992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.043741 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.043761 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.045082 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.045092 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.045434 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.046563 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:01:04.046604 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:01:04.050277 (+ 3673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:01:04.054901 (+ 4624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:04.054914 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:04.054939 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:04.055695 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:04.056188 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:04.056196 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:04.056202 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:04.056529 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:04.057248 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:01:04.057259 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:01:04.057809 (+ 550us) client_negotiation.cc:769] Sending connection context
0808 03:01:04.058231 (+ 422us) client_negotiation.cc:240] Negotiation successful
0808 03:01:04.058917 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":346,"threads_started":1}
I20260808 03:01:04.062642 24118 heartbeater.cc:347] Connected to a master server at 127.0.205.190:36925
I20260808 03:01:04.063056 24118 heartbeater.cc:464] Registering TS with master...
I20260808 03:01:04.064210 24118 heartbeater.cc:511] Master 127.0.205.190:36925 requested a full tablet report, sending...
I20260808 03:01:04.066428 23616 ts_manager.cc:194] Registered new tserver with Master: f9a6ae68adf04408b6b6528648c83295 (127.0.205.130:36127)
I20260808 03:01:04.067848 23616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:53707
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 03:01:04.113998 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36925
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:01:04.478297 24128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:04.478672 24128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:04.478783 24128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:04.486797 24128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:04.486948 24128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:04.487090 24128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:01:04.498061 24128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:04.499495 24128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:04.499960 23909 heartbeater.cc:503] Master 127.0.205.190:36925 was elected leader, sending a full tablet report...
I20260808 03:01:04.501350 24128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:04.516301 24133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:04.516347 24134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:04.516155 24138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:04.517498 24128 server_base.cc:1034] running on GCE node
I20260808 03:01:04.518306 24128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:04.519428 24128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:04.520720 24128 hybrid_clock.cc:648] HybridClock initialized: now 1786158064520662 us; error 66 us; skew 500 ppm
I20260808 03:01:04.521101 24128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:04 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158064, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:04.524778 24128 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:01:04.526701 24128 webserver.cc:460] Webserver started at http://127.0.205.131:42769/ using document root <none> and password file <none>
I20260808 03:01:04.527505 24128 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:04.527660 24128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:04.528085 24128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:01:04.531598 24128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "4bedfcae066e4f84b7a9d9890beb7814"
format_stamp: "Formatted at 2026-08-08 03:01:04 on dist-test-slave-6ntk"
server_key: "3aafa38080e1250662ee6d80701c5329"
server_key_iv: "f71fda32211a9ce0d9855fe347aed333"
server_key_version: "encryptionkey@0"
I20260808 03:01:04.532378 24128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "4bedfcae066e4f84b7a9d9890beb7814"
format_stamp: "Formatted at 2026-08-08 03:01:04 on dist-test-slave-6ntk"
server_key: "3aafa38080e1250662ee6d80701c5329"
server_key_iv: "f71fda32211a9ce0d9855fe347aed333"
server_key_version: "encryptionkey@0"
I20260808 03:01:04.538118 24128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 03:01:04.542002 24156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:04.543560 24128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:01:04.543768 24128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "4bedfcae066e4f84b7a9d9890beb7814"
format_stamp: "Formatted at 2026-08-08 03:01:04 on dist-test-slave-6ntk"
server_key: "3aafa38080e1250662ee6d80701c5329"
server_key_iv: "f71fda32211a9ce0d9855fe347aed333"
server_key_version: "encryptionkey@0"
I20260808 03:01:04.543998 24128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:04.578011 24128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:04.582831 24128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:04.583159 24128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:04.584257 24128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:01:04.585752 24128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:01:04.585829 24128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:04.585988 24128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:01:04.586048 24128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:04.620977 24128 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:40079
I20260808 03:01:04.621027 24320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:40079 every 8 connection(s)
I20260808 03:01:04.622453 24128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:01:04.626209 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 24128
I20260808 03:01:04.626487 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 03:01:04.627036 822 external_mini_cluster.cc:1428] Setting key 108589aaaacb0f2c48c447aa5a367903
Aug 08 03:01:04 dist-test-slave-6ntk krb5kdc[23547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158064, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:01:04.644995 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.625320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:49613 (local address 127.0.205.190:36925)
0808 03:01:04.625824 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.625829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.627025 (+ 1196us) server_negotiation.cc:407] Connection header received
0808 03:01:04.629278 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.629284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.629408 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.629523 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:04.631321 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.632159 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.633735 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.634198 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.634297 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.638050 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:04.638055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:04.638059 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:04.638096 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:04.641126 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:04.642400 (+ 1274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:04.642408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:04.642414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:04.642589 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:04.643186 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:04.643192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:04.643195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:04.643564 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:04.643700 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.644455 (+ 755us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.644653 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":245,"threads_started":1}
I20260808 03:01:04.646127 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.625945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:49613)
0808 03:01:04.626833 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:04.626874 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:04.628920 (+ 2046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:04.629753 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:04.629761 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:04.630504 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:01:04.631154 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.631174 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.632301 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.632305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.633597 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.633610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.634412 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.634418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.634631 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.635366 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:01:04.635391 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:01:04.637764 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:01:04.641326 (+ 3562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:04.641337 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:04.641359 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:04.642165 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:04.642716 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:04.642722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:04.642726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:04.643012 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:04.643691 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:01:04.643699 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:01:04.644094 (+ 395us) client_negotiation.cc:769] Sending connection context
0808 03:01:04.644342 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 03:01:04.644796 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":391,"threads_started":1}
I20260808 03:01:04.647904 24321 heartbeater.cc:347] Connected to a master server at 127.0.205.190:36925
I20260808 03:01:04.648289 24321 heartbeater.cc:464] Registering TS with master...
I20260808 03:01:04.649154 24321 heartbeater.cc:511] Master 127.0.205.190:36925 requested a full tablet report, sending...
I20260808 03:01:04.650875 23616 ts_manager.cc:194] Registered new tserver with Master: 4bedfcae066e4f84b7a9d9890beb7814 (127.0.205.131:40079)
I20260808 03:01:04.651808 23616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:49613
I20260808 03:01:04.655994 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:01:04.674641 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.660797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.0.205.190:36925)
0808 03:01:04.660981 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.660986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.661550 (+ 564us) server_negotiation.cc:407] Connection header received
0808 03:01:04.661656 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.661660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.661742 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.661844 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:04.663422 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.664214 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.665682 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.666182 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.666306 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.669061 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:04.669066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:04.669070 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:04.669105 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:04.672110 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:04.672763 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:04.672767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:04.672769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:04.672897 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:04.673246 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:04.673251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:04.673253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:04.673597 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:04.673815 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.674162 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.674381 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:01:04.679143 23616 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:01:04.686492 23616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:01:04.725976 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.700793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:36127 (local address 127.0.0.1:48120)
0808 03:01:04.713669 (+ 12876us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:04.713738 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:04.714094 (+ 356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:04.716599 (+ 2505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:04.716607 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:04.716652 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:04.717256 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.717276 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.719998 (+ 2722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.720003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.722714 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.722731 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.724707 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.724714 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.724986 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.725060 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 03:01:04.725172 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 03:01:04.725413 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12458,"mutex_wait_us":6,"thread_start_us":12362,"threads_started":1}
I20260808 03:01:04.726467 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.715903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44389 (local address 127.0.0.1:55330)
0808 03:01:04.716938 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:04.716977 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:04.717154 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:04.718511 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:04.718517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:04.718550 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:04.719018 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.719032 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.720909 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.720913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.723301 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.723316 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.725482 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.725488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.725679 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.725761 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 03:01:04.725894 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 03:01:04.726072 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":113,"thread_start_us":1027,"threads_started":1}
I20260808 03:01:04.727283 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.714976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.0.205.129:44389)
0808 03:01:04.717589 (+ 2613us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.717599 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.717628 (+ 29us) server_negotiation.cc:407] Connection header received
0808 03:01:04.717770 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.717777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.718056 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.718252 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:04.719224 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.720732 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.723497 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.724999 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.725122 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.725341 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.726714 (+ 1373us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.726917 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2409,"thread_start_us":2227,"threads_started":1}
I20260808 03:01:04.728259 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.700973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48120 (local address 127.0.205.130:36127)
0808 03:01:04.715740 (+ 14767us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.715751 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.715788 (+ 37us) server_negotiation.cc:407] Connection header received
0808 03:01:04.715909 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.715978 (+ 69us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.716306 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.717513 (+ 1207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:04.717758 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.719839 (+ 2081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.722911 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.724460 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.727254 (+ 2794us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.727648 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.727733 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.727908 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14617,"thread_start_us":14505,"threads_started":1}
I20260808 03:01:04.730031 24342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.701042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:40079 (local address 127.0.0.1:56394)
0808 03:01:04.714962 (+ 13920us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:04.714990 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:04.715158 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:04.717354 (+ 2196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:04.717359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:04.717382 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:04.717916 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.717932 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.721604 (+ 3672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.721610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.725463 (+ 3853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.725481 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.729067 (+ 3586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.729074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.729258 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.729302 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 03:01:04.729388 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 03:01:04.729542 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13725,"mutex_wait_us":1,"thread_start_us":13547,"threads_started":1}
I20260808 03:01:04.730331 24343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.701540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.0.205.131:40079)
0808 03:01:04.713042 (+ 11502us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.713060 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.716126 (+ 3066us) server_negotiation.cc:407] Connection header received
0808 03:01:04.716234 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.716242 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.716446 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.716645 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:04.718193 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.721412 (+ 3219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.725715 (+ 4303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.728514 (+ 2799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.728665 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.728916 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.729586 (+ 670us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.729775 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11183,"thread_start_us":10928,"threads_started":1}
I20260808 03:01:04.731837 23811 tablet_service.cc:1467] Processing CreateTablet for tablet cc174b46362e4edaadb86c60dbd4c3b4 (DEFAULT_TABLE table=test-table [id=d3afa008a927461bbe0f81b9f068d7da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:01:04.733573 23811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc174b46362e4edaadb86c60dbd4c3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:01:04.733940 24023 tablet_service.cc:1467] Processing CreateTablet for tablet cc174b46362e4edaadb86c60dbd4c3b4 (DEFAULT_TABLE table=test-table [id=d3afa008a927461bbe0f81b9f068d7da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:01:04.733940 24221 tablet_service.cc:1467] Processing CreateTablet for tablet cc174b46362e4edaadb86c60dbd4c3b4 (DEFAULT_TABLE table=test-table [id=d3afa008a927461bbe0f81b9f068d7da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:01:04.735356 24023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc174b46362e4edaadb86c60dbd4c3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:01:04.735353 24221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc174b46362e4edaadb86c60dbd4c3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:01:04.750604 24348 tablet_bootstrap.cc:492] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: Bootstrap starting.
I20260808 03:01:04.753186 24349 tablet_bootstrap.cc:492] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: Bootstrap starting.
I20260808 03:01:04.754875 24350 tablet_bootstrap.cc:492] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: Bootstrap starting.
I20260808 03:01:04.755155 24348 tablet_bootstrap.cc:654] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: Neither blocks nor log segments found. Creating new log.
I20260808 03:01:04.757851 24348 log.cc:824] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: Log is configured to *not* fsync() on all Append() calls
I20260808 03:01:04.758385 24349 tablet_bootstrap.cc:654] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: Neither blocks nor log segments found. Creating new log.
I20260808 03:01:04.758894 24350 tablet_bootstrap.cc:654] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: Neither blocks nor log segments found. Creating new log.
I20260808 03:01:04.759512 24349 log.cc:824] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: Log is configured to *not* fsync() on all Append() calls
I20260808 03:01:04.760660 24350 log.cc:824] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: Log is configured to *not* fsync() on all Append() calls
I20260808 03:01:04.763077 24348 tablet_bootstrap.cc:492] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: No bootstrap required, opened a new log
I20260808 03:01:04.763208 24349 tablet_bootstrap.cc:492] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: No bootstrap required, opened a new log
I20260808 03:01:04.763432 24348 ts_tablet_manager.cc:1397] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260808 03:01:04.763553 24349 ts_tablet_manager.cc:1397] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260808 03:01:04.764859 24350 tablet_bootstrap.cc:492] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: No bootstrap required, opened a new log
I20260808 03:01:04.765169 24350 ts_tablet_manager.cc:1397] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260808 03:01:04.771965 24348 raft_consensus.cc:348] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } }
I20260808 03:01:04.772419 24348 raft_consensus.cc:374] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:01:04.772521 24348 raft_consensus.cc:729] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efae2f315c024dabb8fa0bc555cd3147, State: Initialized, Role: FOLLOWER
I20260808 03:01:04.773217 24348 consensus_queue.cc:260] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } }
I20260808 03:01:04.775542 24348 ts_tablet_manager.cc:1428] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: Time spent starting tablet: real 0.012s user 0.008s sys 0.003s
I20260808 03:01:04.776897 24350 raft_consensus.cc:348] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } }
I20260808 03:01:04.777451 24350 raft_consensus.cc:374] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:01:04.777585 24350 raft_consensus.cc:729] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bedfcae066e4f84b7a9d9890beb7814, State: Initialized, Role: FOLLOWER
I20260808 03:01:04.778364 24350 consensus_queue.cc:260] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } }
I20260808 03:01:04.779599 24321 heartbeater.cc:503] Master 127.0.205.190:36925 was elected leader, sending a full tablet report...
I20260808 03:01:04.780829 24350 ts_tablet_manager.cc:1428] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260808 03:01:04.781080 24356 raft_consensus.cc:482] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:01:04.781381 24356 raft_consensus.cc:504] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } }
I20260808 03:01:04.782401 24349 raft_consensus.cc:348] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } }
I20260808 03:01:04.783295 24349 raft_consensus.cc:374] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:01:04.783411 24349 raft_consensus.cc:729] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a6ae68adf04408b6b6528648c83295, State: Initialized, Role: FOLLOWER
I20260808 03:01:04.784116 24349 consensus_queue.cc:260] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } }
I20260808 03:01:04.786074 24356 leader_election.cc:288] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bedfcae066e4f84b7a9d9890beb7814 (127.0.205.131:40079), f9a6ae68adf04408b6b6528648c83295 (127.0.205.130:36127)
I20260808 03:01:04.790766 24118 heartbeater.cc:503] Master 127.0.205.190:36925 was elected leader, sending a full tablet report...
I20260808 03:01:04.791968 24349 ts_tablet_manager.cc:1428] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: Time spent starting tablet: real 0.028s user 0.014s sys 0.007s
I20260808 03:01:04.792599 24358 raft_consensus.cc:482] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:01:04.792989 24358 raft_consensus.cc:504] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } }
I20260808 03:01:04.794720 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.785209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:49999 (local address 127.0.205.130:36127)
0808 03:01:04.785414 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.785421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.787213 (+ 1792us) server_negotiation.cc:407] Connection header received
0808 03:01:04.787485 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.787490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.787569 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.787630 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:04.788566 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.789326 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.792054 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.793262 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.793391 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.793469 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.794309 (+ 840us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.794448 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:01:04.794736 24343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.785068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:42089 (local address 127.0.205.131:40079)
0808 03:01:04.785269 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.785278 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.785302 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:01:04.785596 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.785604 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.785697 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.785779 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:04.787057 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.788193 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.791727 (+ 3534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.793201 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.793377 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.793469 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.794299 (+ 830us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.794467 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:01:04.795356 24358 leader_election.cc:288] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9a6ae68adf04408b6b6528648c83295 (127.0.205.130:36127), efae2f315c024dabb8fa0bc555cd3147 (127.0.205.129:44389)
I20260808 03:01:04.795670 24359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.784491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:40079 (local address 127.0.205.129:42089)
0808 03:01:04.785202 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:04.785226 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:04.785387 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:04.786084 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:04.786088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:04.786120 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:04.786808 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.786836 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.788375 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.788384 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.791462 (+ 3078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.791484 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.793729 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.793736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.794025 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.794078 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 03:01:04.794929 (+ 851us) client_negotiation.cc:240] Negotiation successful
0808 03:01:04.795104 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":77,"thread_start_us":812,"threads_started":1}
I20260808 03:01:04.796155 24360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.784705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:36127 (local address 127.0.205.129:49999)
0808 03:01:04.787089 (+ 2384us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:04.787113 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:04.787326 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:04.787821 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:04.787826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:04.787851 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:04.788361 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.788375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.789466 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.789477 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.791823 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.791838 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.793845 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.793852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.794094 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.794138 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 03:01:04.795260 (+ 1122us) client_negotiation.cc:240] Negotiation successful
0808 03:01:04.795415 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2258,"mutex_wait_us":34,"thread_start_us":2063,"threads_started":1}
I20260808 03:01:04.797499 24056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc174b46362e4edaadb86c60dbd4c3b4" candidate_uuid: "efae2f315c024dabb8fa0bc555cd3147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a6ae68adf04408b6b6528648c83295" is_pre_election: true
I20260808 03:01:04.798131 24056 raft_consensus.cc:2454] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efae2f315c024dabb8fa0bc555cd3147 in term 0.
I20260808 03:01:04.798178 24249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc174b46362e4edaadb86c60dbd4c3b4" candidate_uuid: "efae2f315c024dabb8fa0bc555cd3147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bedfcae066e4f84b7a9d9890beb7814" is_pre_election: true
I20260808 03:01:04.798714 24249 raft_consensus.cc:2454] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efae2f315c024dabb8fa0bc555cd3147 in term 0.
I20260808 03:01:04.799466 23736 leader_election.cc:302] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bedfcae066e4f84b7a9d9890beb7814, efae2f315c024dabb8fa0bc555cd3147; no voters:
I20260808 03:01:04.800084 24356 raft_consensus.cc:2781] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:01:04.800297 24356 raft_consensus.cc:482] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:01:04.800422 24356 raft_consensus.cc:3037] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:01:04.803408 24356 raft_consensus.cc:504] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } }
I20260808 03:01:04.804224 24356 leader_election.cc:288] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [CANDIDATE]: Term 1 election: Requested vote from peers 4bedfcae066e4f84b7a9d9890beb7814 (127.0.205.131:40079), f9a6ae68adf04408b6b6528648c83295 (127.0.205.130:36127)
I20260808 03:01:04.805006 24249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc174b46362e4edaadb86c60dbd4c3b4" candidate_uuid: "efae2f315c024dabb8fa0bc555cd3147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bedfcae066e4f84b7a9d9890beb7814"
I20260808 03:01:04.805233 24056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc174b46362e4edaadb86c60dbd4c3b4" candidate_uuid: "efae2f315c024dabb8fa0bc555cd3147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a6ae68adf04408b6b6528648c83295"
I20260808 03:01:04.805356 24249 raft_consensus.cc:3037] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:01:04.805478 24056 raft_consensus.cc:3037] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:01:04.807628 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.796143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:36127 (local address 127.0.205.131:48977)
0808 03:01:04.796345 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:04.796368 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:04.796514 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:04.799874 (+ 3360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:04.799879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:04.799909 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:04.800118 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.800138 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.802247 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.802257 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.804343 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.804361 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.806719 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.806730 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.807000 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.807042 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 03:01:04.807147 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 03:01:04.807309 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 03:01:04.808291 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.799302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:48977 (local address 127.0.205.130:36127)
0808 03:01:04.799564 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.799569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.799587 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:01:04.799649 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.799653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.799723 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.799850 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:04.800379 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.802040 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.804935 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.806305 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.806530 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.806610 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.807860 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.808018 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260808 03:01:04.808969 24055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc174b46362e4edaadb86c60dbd4c3b4" candidate_uuid: "4bedfcae066e4f84b7a9d9890beb7814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a6ae68adf04408b6b6528648c83295" is_pre_election: true
I20260808 03:01:04.809794 24249 raft_consensus.cc:2454] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efae2f315c024dabb8fa0bc555cd3147 in term 1.
I20260808 03:01:04.810117 24056 raft_consensus.cc:2454] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efae2f315c024dabb8fa0bc555cd3147 in term 1.
I20260808 03:01:04.810452 23736 leader_election.cc:302] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bedfcae066e4f84b7a9d9890beb7814, efae2f315c024dabb8fa0bc555cd3147; no voters:
I20260808 03:01:04.810801 24356 raft_consensus.cc:2781] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:01:04.810989 24356 raft_consensus.cc:686] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 1 LEADER]: Becoming Leader. State: Replica: efae2f315c024dabb8fa0bc555cd3147, State: Running, Role: LEADER
I20260808 03:01:04.811545 24356 consensus_queue.cc:237] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } }
I20260808 03:01:04.814116 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.797252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:44451 (local address 127.0.205.129:44389)
0808 03:01:04.797474 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.797482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.807086 (+ 9604us) server_negotiation.cc:407] Connection header received
0808 03:01:04.807616 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.807622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.807731 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.807810 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:04.808738 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.809852 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.812132 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.813024 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.813239 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.813324 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.813656 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.813814 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:01:04.814311 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.796551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44389 (local address 127.0.205.131:44451)
0808 03:01:04.806957 (+ 10406us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:04.806981 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:04.807410 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:04.808052 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:04.808057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:04.808092 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:04.808475 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.808488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.810138 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.810147 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.811959 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:04.811973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.813249 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.813256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:04.813437 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.813464 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 03:01:04.813595 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 03:01:04.813779 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10232,"thread_start_us":10548,"threads_started":1}
I20260808 03:01:04.815544 23839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc174b46362e4edaadb86c60dbd4c3b4" candidate_uuid: "4bedfcae066e4f84b7a9d9890beb7814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efae2f315c024dabb8fa0bc555cd3147" is_pre_election: true
I20260808 03:01:04.816797 24161 leader_election.cc:302] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bedfcae066e4f84b7a9d9890beb7814; no voters: efae2f315c024dabb8fa0bc555cd3147, f9a6ae68adf04408b6b6528648c83295
I20260808 03:01:04.817257 24358 raft_consensus.cc:2726] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 03:01:04.816629 23615 catalog_manager.cc:5495] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 reported cstate change: term changed from 0 to 1, leader changed from <none> to efae2f315c024dabb8fa0bc555cd3147 (127.0.205.129). New cstate: current_term: 1 leader_uuid: "efae2f315c024dabb8fa0bc555cd3147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } health_report { overall_health: HEALTHY } } }
I20260808 03:01:04.856112 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.850832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.0.205.129:44389)
0808 03:01:04.851013 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.851017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.851126 (+ 109us) server_negotiation.cc:407] Connection header received
0808 03:01:04.851298 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.851302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.851399 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.851535 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:01:04.851884 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.852646 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.854130 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.854542 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.854637 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.855081 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:01:04.855613 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:01:04.855728 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.855799 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.855868 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:01:04.865218 24056 raft_consensus.cc:1261] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295 [term 1 FOLLOWER]: Refusing update from remote peer efae2f315c024dabb8fa0bc555cd3147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:01:04.865283 24249 raft_consensus.cc:1261] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 1 FOLLOWER]: Refusing update from remote peer efae2f315c024dabb8fa0bc555cd3147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:01:04.866430 24357 consensus_queue.cc:1035] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:01:04.866832 24356 consensus_queue.cc:1035] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 03:01:04.874219 24324 tablet.cc:2369] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:01:04.898579 23615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:01:04.898914 23615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:01:04.902812 23615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Sending DeleteTablet for 3 replicas of tablet cc174b46362e4edaadb86c60dbd4c3b4
I20260808 03:01:04.904203 24221 tablet_service.cc:1514] Processing DeleteTablet for tablet cc174b46362e4edaadb86c60dbd4c3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:01:04 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:56394
I20260808 03:01:04.904281 24023 tablet_service.cc:1514] Processing DeleteTablet for tablet cc174b46362e4edaadb86c60dbd4c3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:01:04 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:48120
I20260808 03:01:04.904436 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 23709
I20260808 03:01:04.905457 23811 tablet_service.cc:1514] Processing DeleteTablet for tablet cc174b46362e4edaadb86c60dbd4c3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:01:04 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:55330
I20260808 03:01:04.916038 24393 tablet_replica.cc:330] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: stopping tablet replica
I20260808 03:01:04.916750 24393 raft_consensus.cc:2229] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:01:04.917490 24393 raft_consensus.cc:2258] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:01:04.918452 24395 tablet_replica.cc:330] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: stopping tablet replica
I20260808 03:01:04.918510 24392 tablet_replica.cc:330] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: stopping tablet replica
I20260808 03:01:04.918928 24392 raft_consensus.cc:2229] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:01:04.918941 24395 raft_consensus.cc:2229] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:01:04.919447 24392 raft_consensus.cc:2258] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:01:04.921818 24392 ts_tablet_manager.cc:1905] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:01:04.921922 24393 ts_tablet_manager.cc:1905] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:01:04.919735 24395 raft_consensus.cc:2258] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:01:04.926222 24395 ts_tablet_manager.cc:1905] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:01:04.929363 24392 ts_tablet_manager.cc:1918] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:01:04.929541 24392 log.cc:1196] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/cc174b46362e4edaadb86c60dbd4c3b4
I20260808 03:01:04.930071 24393 ts_tablet_manager.cc:1918] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:01:04.930148 24392 ts_tablet_manager.cc:1939] T cc174b46362e4edaadb86c60dbd4c3b4 P 4bedfcae066e4f84b7a9d9890beb7814: Deleting consensus metadata
I20260808 03:01:04.930389 24393 log.cc:1196] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/cc174b46362e4edaadb86c60dbd4c3b4
I20260808 03:01:04.931063 24393 ts_tablet_manager.cc:1939] T cc174b46362e4edaadb86c60dbd4c3b4 P f9a6ae68adf04408b6b6528648c83295: Deleting consensus metadata
I20260808 03:01:04.932307 23591 catalog_manager.cc:4841] TS 4bedfcae066e4f84b7a9d9890beb7814 (127.0.205.131:40079): tablet cc174b46362e4edaadb86c60dbd4c3b4 (table test-table [id=d3afa008a927461bbe0f81b9f068d7da]) successfully deleted
I20260808 03:01:04.932973 23592 catalog_manager.cc:4841] TS f9a6ae68adf04408b6b6528648c83295 (127.0.205.130:36127): tablet cc174b46362e4edaadb86c60dbd4c3b4 (table test-table [id=d3afa008a927461bbe0f81b9f068d7da]) successfully deleted
I20260808 03:01:04.933991 24395 ts_tablet_manager.cc:1918] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:01:04.934183 24395 log.cc:1196] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/cc174b46362e4edaadb86c60dbd4c3b4
I20260808 03:01:04.934837 24395 ts_tablet_manager.cc:1939] T cc174b46362e4edaadb86c60dbd4c3b4 P efae2f315c024dabb8fa0bc555cd3147: Deleting consensus metadata
I20260808 03:01:04.936832 23592 catalog_manager.cc:4841] TS efae2f315c024dabb8fa0bc555cd3147 (127.0.205.129:44389): tablet cc174b46362e4edaadb86c60dbd4c3b4 (table test-table [id=d3afa008a927461bbe0f81b9f068d7da]) successfully deleted
Aug 08 03:01:04 dist-test-slave-6ntk krb5kdc[23547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 03:01:04.948925 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:04.905308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.0.205.129:44389)
0808 03:01:04.905539 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:04.905545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:04.921069 (+ 15524us) server_negotiation.cc:407] Connection header received
0808 03:01:04.921420 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:04.921426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:04.921515 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:04.921595 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:04.932967 (+ 11372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.934186 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.936637 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:04.937189 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:04.937371 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:04.942143 (+ 4772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:04.942159 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:04.942176 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:04.942221 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:04.945600 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:04.946508 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:04.946516 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:04.946518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:04.946688 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:04.947121 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:04.947125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:04.947127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:04.947598 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:04.947997 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:04.948270 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 03:01:04.948542 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 03:01:04.949707 23897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:05.064369 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 23709
I20260808 03:01:05.093519 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 23915
Aug 08 03:01:05 dist-test-slave-6ntk krb5kdc[23547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 03:01:05.110543 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:05.094667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.0.205.130:36127)
0808 03:01:05.094914 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:05.094920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:05.094941 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:01:05.095012 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:05.095023 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:05.095097 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:05.095210 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:05.096672 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:05.097409 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:05.098932 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:05.099356 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:05.099499 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:05.104299 (+ 4800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:05.104317 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:05.104335 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:05.104376 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:05.107508 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:05.108269 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:05.108277 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:05.108279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:05.108424 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:05.108841 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:05.108850 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:05.108852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:05.109228 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:05.109569 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:05.109944 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 03:01:05.110181 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 03:01:05.111270 24112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:05.203415 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 23915
I20260808 03:01:05.228569 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 24128
Aug 08 03:01:05 dist-test-slave-6ntk krb5kdc[23547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 03:01:05.251000 24343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:05.229369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.0.205.131:40079)
0808 03:01:05.229559 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:05.229567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:05.229616 (+ 49us) server_negotiation.cc:407] Connection header received
0808 03:01:05.229864 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:05.229906 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:05.229984 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:05.230107 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:05.231684 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:05.232456 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:05.233810 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:05.234418 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:05.234557 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:05.244287 (+ 9730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:05.244309 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:05.244328 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:05.244374 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:05.247567 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:05.248434 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:05.248443 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:05.248445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:05.248639 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:05.249114 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:05.249117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:05.249119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:05.249519 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:05.249744 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:05.250333 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 03:01:05.250614 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:01:05.251688 24314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:05.346498 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 24128
I20260808 03:01:05.373564 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 23571
I20260808 03:01:05.374902 23667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:05.490293 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 23571
I20260808 03:01:25.522130 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:36925
--webserver_interface=127.0.205.190
--webserver_port=35623
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:36925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:01:25.890405 24403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:25.890805 24403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:25.891000 24403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:25.898504 24403 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 03:01:25.898675 24403 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 03:01:25.898782 24403 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 03:01:25.898885 24403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:01:25.898972 24403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:25.899073 24403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:25.899164 24403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:01:25.899255 24403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:01:25.909657 24403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:36925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:36925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=35623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:25.911123 24403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:25.912513 24403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:25.941212 24418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:25.941628 24403 server_base.cc:1034] running on GCE node
W20260808 03:01:25.941361 24413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:25.941331 24414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:25.943131 24403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:25.944813 24403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:25.946038 24403 hybrid_clock.cc:648] HybridClock initialized: now 1786158085945954 us; error 38 us; skew 500 ppm
I20260808 03:01:25.946486 24403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:25 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158085, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:25.950332 24403 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:01:25.952157 24403 webserver.cc:460] Webserver started at http://127.0.205.190:35623/ using document root <none> and password file <none>
I20260808 03:01:25.953011 24403 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:25.953174 24403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:25.959623 24403 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:01:25.962846 24429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:25.964341 24403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:01:25.964562 24403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "ada042949a114aef932790d2e07a68a1"
format_stamp: "Formatted at 2026-08-08 03:01:02 on dist-test-slave-6ntk"
server_key: "095c939b18ae905754a6f75a181a093d"
server_key_iv: "6ccb5f22f385859f5816602c385cf3f7"
server_key_version: "encryptionkey@0"
I20260808 03:01:25.965662 24403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:25.982905 24403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:25.987708 24403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:25.988056 24403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:26.008896 24403 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:36925
I20260808 03:01:26.008921 24500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:36925 every 8 connection(s)
I20260808 03:01:26.010672 24403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:01:26.014592 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 24403
I20260808 03:01:26.015481 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:44389
--local_ip_for_outbound_sockets=127.0.205.129
--tserver_master_addrs=127.0.205.190:36925
--webserver_port=42981
--webserver_interface=127.0.205.129
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:01:26.019058 24501 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:01:26.022962 24501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Bootstrap starting.
I20260808 03:01:26.045065 24501 log.cc:824] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Log is configured to *not* fsync() on all Append() calls
I20260808 03:01:26.059195 24501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:01:26.059867 24501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Bootstrap complete.
I20260808 03:01:26.071904 24501 raft_consensus.cc:348] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:26.072901 24501 raft_consensus.cc:729] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada042949a114aef932790d2e07a68a1, State: Initialized, Role: FOLLOWER
I20260808 03:01:26.073617 24501 consensus_queue.cc:260] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:26.073928 24501 raft_consensus.cc:388] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:01:26.074098 24501 raft_consensus.cc:482] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:01:26.074328 24501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 1 FOLLOWER]: Advancing to term 2
I20260808 03:01:26.078866 24501 raft_consensus.cc:504] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:26.079530 24501 leader_election.cc:302] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ada042949a114aef932790d2e07a68a1; no voters:
I20260808 03:01:26.080302 24501 leader_election.cc:288] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 03:01:26.080472 24507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 03:01:26.081804 24507 raft_consensus.cc:686] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 2 LEADER]: Becoming Leader. State: Replica: ada042949a114aef932790d2e07a68a1, State: Running, Role: LEADER
I20260808 03:01:26.082540 24507 consensus_queue.cc:237] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:26.082999 24501 sys_catalog.cc:564] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:01:26.085471 24509 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ada042949a114aef932790d2e07a68a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } } }
I20260808 03:01:26.086043 24509 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:01:26.090731 24511 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ada042949a114aef932790d2e07a68a1. Latest consensus state: current_term: 2 leader_uuid: "ada042949a114aef932790d2e07a68a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } } }
I20260808 03:01:26.097707 24511 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:01:26.099560 24516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:01:26.110634 24516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:01:26.112349 24516 catalog_manager.cc:1259] Loaded cluster ID: 68b418f9dfb043c2bb0de8afcf8b0a49
I20260808 03:01:26.112613 24516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:01:26.117291 24516 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:01:26.119464 24516 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Loaded TSK: 0
I20260808 03:01:26.151609 24516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Generated new TSK 1
I20260808 03:01:26.152060 24516 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 03:01:26.407202 24505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:26.407524 24505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:26.407702 24505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:26.416152 24505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:26.416280 24505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:26.416527 24505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:01:26.427466 24505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:44389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=42981
--tserver_master_addrs=127.0.205.190:36925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:26.428879 24505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:26.430353 24505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:26.445770 24543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:26.446695 24539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:26.447327 24505 server_base.cc:1034] running on GCE node
W20260808 03:01:26.447350 24545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:26.448272 24505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:26.449645 24505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:26.450927 24505 hybrid_clock.cc:648] HybridClock initialized: now 1786158086450820 us; error 91 us; skew 500 ppm
I20260808 03:01:26.451339 24505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:26 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158086, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:26.455801 24505 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:01:26.457752 24505 webserver.cc:460] Webserver started at http://127.0.205.129:42981/ using document root <none> and password file <none>
I20260808 03:01:26.458734 24505 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:26.458920 24505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:26.466758 24505 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 03:01:26.470364 24553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:26.472128 24505 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260808 03:01:26.472318 24505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "efae2f315c024dabb8fa0bc555cd3147"
format_stamp: "Formatted at 2026-08-08 03:01:03 on dist-test-slave-6ntk"
server_key: "8c76dbe72c9198b0feff4d914e742061"
server_key_iv: "c63174d185cde5643b18801793baa992"
server_key_version: "encryptionkey@0"
I20260808 03:01:26.473716 24505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:26.505573 24505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:26.510562 24505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:26.510888 24505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:26.512048 24505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:01:26.513617 24505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:01:26.513712 24505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:26.513867 24505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:01:26.514007 24505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 03:01:26.550369 24505 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:44389
I20260808 03:01:26.550438 24741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:44389 every 8 connection(s)
I20260808 03:01:26.551913 24505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:01:26.558144 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 24505
I20260808 03:01:26.559011 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:36127
--local_ip_for_outbound_sockets=127.0.205.130
--tserver_master_addrs=127.0.205.190:36925
--webserver_port=44889
--webserver_interface=127.0.205.130
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:01:26 dist-test-slave-6ntk krb5kdc[23547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158086, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:01:26.579674 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:26.555304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:33607)
0808 03:01:26.556177 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:26.556223 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:26.558593 (+ 2370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:26.560257 (+ 1664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:26.560272 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:26.561167 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:01:26.562016 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:26.562049 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:26.563779 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:26.563787 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:26.565987 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:26.566006 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:26.567064 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:26.567074 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:26.567397 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:26.568213 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:01:26.568248 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:01:26.571212 (+ 2964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:01:26.574976 (+ 3764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:26.574985 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:26.575003 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:26.575474 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:26.575965 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:26.575971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:26.575974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:26.576183 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:26.577009 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:01:26.577018 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:01:26.577380 (+ 362us) client_negotiation.cc:769] Sending connection context
0808 03:01:26.577665 (+ 285us) client_negotiation.cc:240] Negotiation successful
0808 03:01:26.578293 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":384,"threads_started":1}
I20260808 03:01:26.579675 24745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:26.555153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:33607 (local address 127.0.205.190:36925)
0808 03:01:26.555811 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:26.555825 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:26.556357 (+ 532us) server_negotiation.cc:407] Connection header received
0808 03:01:26.559282 (+ 2925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:26.559301 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:26.559689 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:26.560007 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:26.562262 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:26.563569 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:26.566135 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:26.566805 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:26.566938 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:26.571542 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:26.571551 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:26.571566 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:26.571605 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:26.574802 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:26.575644 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:26.575651 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:26.575657 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:26.575829 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:26.576314 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:26.576324 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:26.576327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:26.576855 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:26.577077 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:26.577939 (+ 862us) server_negotiation.cc:299] Negotiation successful
0808 03:01:26.578358 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":292,"threads_started":1}
I20260808 03:01:26.583211 24742 heartbeater.cc:347] Connected to a master server at 127.0.205.190:36925
I20260808 03:01:26.583735 24742 heartbeater.cc:464] Registering TS with master...
I20260808 03:01:26.584761 24742 heartbeater.cc:511] Master 127.0.205.190:36925 requested a full tablet report, sending...
I20260808 03:01:26.587815 24450 ts_manager.cc:194] Registered new tserver with Master: efae2f315c024dabb8fa0bc555cd3147 (127.0.205.129:44389)
I20260808 03:01:26.590426 24450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:33607
W20260808 03:01:26.917829 24748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:26.918174 24748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:26.918280 24748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:26.926416 24748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:26.926542 24748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:26.926661 24748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:01:26.937150 24748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:36127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=44889
--tserver_master_addrs=127.0.205.190:36925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:26.938586 24748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:26.939888 24748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:26.954093 24756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:26.954147 24755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:26.954321 24760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:26.955152 24748 server_base.cc:1034] running on GCE node
I20260808 03:01:26.955819 24748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:26.956880 24748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:26.958123 24748 hybrid_clock.cc:648] HybridClock initialized: now 1786158086958057 us; error 54 us; skew 500 ppm
I20260808 03:01:26.958495 24748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:26 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158086, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:26.961942 24748 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:01:26.963383 24748 webserver.cc:460] Webserver started at http://127.0.205.130:44889/ using document root <none> and password file <none>
I20260808 03:01:26.964059 24748 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:26.964182 24748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:26.969612 24748 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:01:26.972679 24767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:26.974169 24748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:01:26.974308 24748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "f9a6ae68adf04408b6b6528648c83295"
format_stamp: "Formatted at 2026-08-08 03:01:03 on dist-test-slave-6ntk"
server_key: "077baba89221530f917432ccca35017d"
server_key_iv: "8eedfe2c5156cd9ecaa0785ef300b16b"
server_key_version: "encryptionkey@0"
I20260808 03:01:26.975236 24748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:26.989491 24748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:26.994072 24748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:26.994313 24748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:26.995239 24748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:01:26.996523 24748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:01:26.996609 24748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:26.996688 24748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:01:26.996744 24748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:27.033440 24748 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:36127
I20260808 03:01:27.033543 24947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:36127 every 8 connection(s)
I20260808 03:01:27.034992 24748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:01:27.043203 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 24748
I20260808 03:01:27.043867 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:40079
--local_ip_for_outbound_sockets=127.0.205.131
--tserver_master_addrs=127.0.205.190:36925
--webserver_port=42769
--webserver_interface=127.0.205.131
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 03:01:27 dist-test-slave-6ntk krb5kdc[23547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158086, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:01:27.056421 24745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.037730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:49355 (local address 127.0.205.190:36925)
0808 03:01:27.037991 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.037996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.039374 (+ 1378us) server_negotiation.cc:407] Connection header received
0808 03:01:27.041106 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.041110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.041189 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.041279 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:27.043176 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.043945 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.045567 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.046099 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.046238 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.049862 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:27.049915 (+ 53us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:27.049920 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:27.049963 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:27.052663 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:27.053428 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:27.053435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:27.053438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:27.053574 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:27.054197 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:27.054201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:27.054203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:27.054594 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:27.054827 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:27.055697 (+ 870us) server_negotiation.cc:299] Negotiation successful
0808 03:01:27.055973 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:01:27.057459 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.038313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:49355)
0808 03:01:27.039192 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:27.039232 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:27.040849 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:27.041493 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:27.041502 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:27.042174 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:01:27.042944 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.042970 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.044167 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.044171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.045410 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.045422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.046349 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.046367 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.046593 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.047323 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:01:27.047347 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:01:27.049570 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:01:27.052811 (+ 3241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:27.052821 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:27.052840 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:27.053287 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:27.053752 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:27.053760 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:27.053766 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:27.054076 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:27.054829 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:01:27.054837 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:01:27.055245 (+ 408us) client_negotiation.cc:769] Sending connection context
0808 03:01:27.055543 (+ 298us) client_negotiation.cc:240] Negotiation successful
0808 03:01:27.056030 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":271,"threads_started":1}
I20260808 03:01:27.059283 24949 heartbeater.cc:347] Connected to a master server at 127.0.205.190:36925
I20260808 03:01:27.059652 24949 heartbeater.cc:464] Registering TS with master...
I20260808 03:01:27.060451 24949 heartbeater.cc:511] Master 127.0.205.190:36925 requested a full tablet report, sending...
I20260808 03:01:27.062110 24450 ts_manager.cc:194] Registered new tserver with Master: f9a6ae68adf04408b6b6528648c83295 (127.0.205.130:36127)
I20260808 03:01:27.062963 24450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:49355
W20260808 03:01:27.397843 24953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:27.398269 24953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:27.398455 24953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:27.406170 24953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:27.406333 24953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:27.406515 24953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:01:27.416754 24953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:40079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=42769
--tserver_master_addrs=127.0.205.190:36925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:27.418187 24953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:27.419529 24953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:27.433104 24961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:27.433215 24963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:27.433087 24965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:27.433894 24953 server_base.cc:1034] running on GCE node
I20260808 03:01:27.434638 24953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:27.435612 24953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:27.436853 24953 hybrid_clock.cc:648] HybridClock initialized: now 1786158087436812 us; error 56 us; skew 500 ppm
I20260808 03:01:27.437191 24953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:27 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158087, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:27.440531 24953 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:01:27.442293 24953 webserver.cc:460] Webserver started at http://127.0.205.131:42769/ using document root <none> and password file <none>
I20260808 03:01:27.443079 24953 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:27.443223 24953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:27.449431 24953 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 03:01:27.452457 24978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:27.453965 24953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 03:01:27.454176 24953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "4bedfcae066e4f84b7a9d9890beb7814"
format_stamp: "Formatted at 2026-08-08 03:01:04 on dist-test-slave-6ntk"
server_key: "3aafa38080e1250662ee6d80701c5329"
server_key_iv: "f71fda32211a9ce0d9855fe347aed333"
server_key_version: "encryptionkey@0"
I20260808 03:01:27.455299 24953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:27.479216 24953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:27.483595 24953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:27.483918 24953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:27.485018 24953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:01:27.486562 24953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:01:27.486666 24953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:27.486837 24953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:01:27.486948 24953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:27.521327 24953 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:40079
I20260808 03:01:27.521392 25162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:40079 every 8 connection(s)
I20260808 03:01:27.522953 24953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:01:27.526386 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 24953
Aug 08 03:01:27 dist-test-slave-6ntk krb5kdc[23547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158087, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:01:27.544015 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.527571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37392 (local address 127.0.205.190:36925)
0808 03:01:27.528080 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.528087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.528447 (+ 360us) server_negotiation.cc:407] Connection header received
0808 03:01:27.528691 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.528695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.528757 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.528874 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:27.530805 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.531569 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.532994 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.533438 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.533579 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.536765 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:27.536770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:27.536773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:27.536831 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:27.540888 (+ 4057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:27.541648 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:27.541653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:27.541656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:27.541824 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:27.542287 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:27.542292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:27.542296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:27.542732 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:27.543012 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:27.543326 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 03:01:27.543602 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":260,"threads_started":1}
I20260808 03:01:27.545117 24745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.525792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:36525 (local address 127.0.205.190:36925)
0808 03:01:27.526082 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.526090 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.527377 (+ 1287us) server_negotiation.cc:407] Connection header received
0808 03:01:27.529619 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.529624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.529702 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.529841 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:27.531440 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.532208 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.533794 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.534352 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.534492 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.538124 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:27.538130 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:27.538134 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:27.538210 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:27.541637 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:27.542448 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:27.542453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:27.542456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:27.542572 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:27.543069 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:27.543074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:27.543076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:27.543323 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:27.543555 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:27.544553 (+ 998us) server_negotiation.cc:299] Negotiation successful
0808 03:01:27.544740 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260808 03:01:27.546702 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.526281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:36525)
0808 03:01:27.527193 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:27.527252 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:27.529309 (+ 2057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:27.530126 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:27.530135 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:27.530676 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:01:27.531252 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.531270 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.532385 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.532395 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.533637 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.533649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.534619 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.534627 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.534858 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.535542 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:01:27.535569 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:01:27.537824 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:01:27.541783 (+ 3959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:27.541791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:27.541805 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:27.542324 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:27.542701 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:01:27.542704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:01:27.542706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:01:27.542923 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:01:27.543475 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:01:27.543485 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:01:27.544025 (+ 540us) client_negotiation.cc:769] Sending connection context
0808 03:01:27.544442 (+ 417us) client_negotiation.cc:240] Negotiation successful
0808 03:01:27.545086 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":351,"threads_started":1}
I20260808 03:01:27.548251 25164 heartbeater.cc:347] Connected to a master server at 127.0.205.190:36925
I20260808 03:01:27.548609 25164 heartbeater.cc:464] Registering TS with master...
I20260808 03:01:27.549361 25164 heartbeater.cc:511] Master 127.0.205.190:36925 requested a full tablet report, sending...
I20260808 03:01:27.551076 24450 ts_manager.cc:194] Registered new tserver with Master: 4bedfcae066e4f84b7a9d9890beb7814 (127.0.205.131:40079)
I20260808 03:01:27.551975 24450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:36525
I20260808 03:01:27.560084 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:01:27.566857 24745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.561022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37396 (local address 127.0.205.190:36925)
0808 03:01:27.561187 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.561191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.561763 (+ 572us) server_negotiation.cc:407] Connection header received
0808 03:01:27.561999 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.562003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.562156 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.562288 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:01:27.562717 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.563534 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.564990 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.565397 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.565488 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.566042 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:01:27.566341 (+ 299us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 03:01:27.566657 (+ 316us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37396: token expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 03:01:27.567015 24745 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37396: token expired
W20260808 03:01:27.567647 24329 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.205.190:36925)
I20260808 03:01:27.581545 24745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.568189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.0.205.190:36925)
0808 03:01:27.568336 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.568339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.568354 (+ 15us) server_negotiation.cc:407] Connection header received
0808 03:01:27.568482 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.568485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.568553 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.568628 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:27.570129 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.570957 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.572325 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.572716 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.572808 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.575647 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:27.575651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:27.575655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:27.575696 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:27.578564 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:27.579530 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:27.579534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:27.579536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:27.579706 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:27.580191 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:27.580196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:27.580199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:27.580574 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:27.580759 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:27.581095 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 03:01:27.581286 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 03:01:27.585531 24450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:01:27.593056 24450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:01:27.595000 24742 heartbeater.cc:503] Master 127.0.205.190:36925 was elected leader, sending a full tablet report...
I20260808 03:01:27.618561 25184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.605843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:36127 (local address 127.0.0.1:45874)
0808 03:01:27.606710 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:27.606748 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:27.606944 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:27.608097 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:27.608108 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:27.608155 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:27.608706 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.608723 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.611407 (+ 2684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.611416 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.614861 (+ 3445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.614898 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.617323 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.617330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.617599 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.617666 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 03:01:27.617824 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 03:01:27.618089 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":407,"threads_started":1}
I20260808 03:01:27.619804 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.606085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45874 (local address 127.0.205.130:36127)
0808 03:01:27.606681 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.606690 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.606873 (+ 183us) server_negotiation.cc:407] Connection header received
0808 03:01:27.607191 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.607201 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.607493 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.607718 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:27.608972 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.611175 (+ 2203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.615154 (+ 3979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.617000 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.617177 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.617414 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:27.619123 (+ 1709us) server_negotiation.cc:299] Negotiation successful
0808 03:01:27.619369 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":310,"threads_started":1}
I20260808 03:01:27.624926 24849 tablet_service.cc:1467] Processing CreateTablet for tablet 6a698e40d1de45e18df3175ce15a0a98 (DEFAULT_TABLE table=test-table [id=92da85b26837420abb74fc5886eaa82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:01:27.626809 25187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.607141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44389 (local address 127.0.0.1:47538)
0808 03:01:27.616851 (+ 9710us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:27.616878 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:27.617042 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:27.619503 (+ 2461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:27.619510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:27.619540 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:27.619978 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.619989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.622252 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.622256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.623972 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.623989 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.625800 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.625807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.626050 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.626094 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 03:01:27.626205 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 03:01:27.626345 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9494,"mutex_wait_us":130,"thread_start_us":11940,"threads_started":1}
I20260808 03:01:27.627406 24849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a698e40d1de45e18df3175ce15a0a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:01:27.627671 25188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.607659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.0.205.129:44389)
0808 03:01:27.618534 (+ 10875us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.618545 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.618581 (+ 36us) server_negotiation.cc:407] Connection header received
0808 03:01:27.618700 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.618711 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.619013 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.619255 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:27.620325 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.622039 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.624150 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.625562 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.625738 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.626027 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:27.626277 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 03:01:27.627314 (+ 1037us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10688,"thread_start_us":10531,"threads_started":1}
I20260808 03:01:27.628284 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.608254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.0.205.131:40079)
0808 03:01:27.617999 (+ 9745us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.618008 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.619103 (+ 1095us) server_negotiation.cc:407] Connection header received
0808 03:01:27.619222 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.619229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.619476 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.619672 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:27.620396 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.622047 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.624861 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.626591 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.626803 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.627105 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:27.627763 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 03:01:27.627941 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9510,"thread_start_us":9361,"threads_started":1}
I20260808 03:01:27.628300 25184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.607272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:40079 (local address 127.0.0.1:37928)
0808 03:01:27.618850 (+ 11578us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:27.618868 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:27.618975 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:27.619983 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:27.619987 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:27.620015 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:27.620191 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.620205 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.622203 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.622211 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.624599 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.624618 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.627260 (+ 2642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.627268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.627520 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.627565 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 03:01:27.627709 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 03:01:27.627893 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11486,"mutex_wait_us":1,"thread_start_us":9188,"threads_started":1}
I20260808 03:01:27.632073 24627 tablet_service.cc:1467] Processing CreateTablet for tablet 6a698e40d1de45e18df3175ce15a0a98 (DEFAULT_TABLE table=test-table [id=92da85b26837420abb74fc5886eaa82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:01:27.632838 25048 tablet_service.cc:1467] Processing CreateTablet for tablet 6a698e40d1de45e18df3175ce15a0a98 (DEFAULT_TABLE table=test-table [id=92da85b26837420abb74fc5886eaa82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:01:27.634421 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a698e40d1de45e18df3175ce15a0a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:01:27.634702 25048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a698e40d1de45e18df3175ce15a0a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:01:27.647291 25190 tablet_bootstrap.cc:492] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: Bootstrap starting.
I20260808 03:01:27.650002 25191 tablet_bootstrap.cc:492] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: Bootstrap starting.
I20260808 03:01:27.651717 25190 tablet_bootstrap.cc:654] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: Neither blocks nor log segments found. Creating new log.
I20260808 03:01:27.653330 25190 log.cc:824] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: Log is configured to *not* fsync() on all Append() calls
I20260808 03:01:27.653848 25194 tablet_bootstrap.cc:492] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: Bootstrap starting.
I20260808 03:01:27.654907 25191 tablet_bootstrap.cc:654] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: Neither blocks nor log segments found. Creating new log.
I20260808 03:01:27.656451 25191 log.cc:824] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: Log is configured to *not* fsync() on all Append() calls
I20260808 03:01:27.657830 25194 tablet_bootstrap.cc:654] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: Neither blocks nor log segments found. Creating new log.
I20260808 03:01:27.658099 25190 tablet_bootstrap.cc:492] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: No bootstrap required, opened a new log
I20260808 03:01:27.658502 25190 ts_tablet_manager.cc:1397] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260808 03:01:27.659492 25194 log.cc:824] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: Log is configured to *not* fsync() on all Append() calls
I20260808 03:01:27.660506 25191 tablet_bootstrap.cc:492] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: No bootstrap required, opened a new log
I20260808 03:01:27.660894 25191 ts_tablet_manager.cc:1397] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260808 03:01:27.663587 25194 tablet_bootstrap.cc:492] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: No bootstrap required, opened a new log
I20260808 03:01:27.663920 25194 ts_tablet_manager.cc:1397] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260808 03:01:27.669987 25191 raft_consensus.cc:348] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } } peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } }
I20260808 03:01:27.670569 25191 raft_consensus.cc:374] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:01:27.670703 25191 raft_consensus.cc:729] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efae2f315c024dabb8fa0bc555cd3147, State: Initialized, Role: FOLLOWER
I20260808 03:01:27.671001 25190 raft_consensus.cc:348] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } } peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } }
I20260808 03:01:27.671492 25190 raft_consensus.cc:374] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:01:27.671608 25190 raft_consensus.cc:729] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a6ae68adf04408b6b6528648c83295, State: Initialized, Role: FOLLOWER
I20260808 03:01:27.671514 25191 consensus_queue.cc:260] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } } peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } }
I20260808 03:01:27.672394 25190 consensus_queue.cc:260] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } } peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } }
I20260808 03:01:27.675376 25191 ts_tablet_manager.cc:1428] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260808 03:01:27.676779 25194 raft_consensus.cc:348] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } } peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } }
I20260808 03:01:27.677371 25194 raft_consensus.cc:374] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:01:27.677505 25194 raft_consensus.cc:729] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bedfcae066e4f84b7a9d9890beb7814, State: Initialized, Role: FOLLOWER
I20260808 03:01:27.678450 25194 consensus_queue.cc:260] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } } peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } }
I20260808 03:01:27.680238 25164 heartbeater.cc:503] Master 127.0.205.190:36925 was elected leader, sending a full tablet report...
I20260808 03:01:27.680855 25194 ts_tablet_manager.cc:1428] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260808 03:01:27.684065 24949 heartbeater.cc:503] Master 127.0.205.190:36925 was elected leader, sending a full tablet report...
I20260808 03:01:27.685196 25190 ts_tablet_manager.cc:1428] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: Time spent starting tablet: real 0.027s user 0.013s sys 0.007s
W20260808 03:01:27.774727 25165 tablet.cc:2369] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:01:27.787067 24950 tablet.cc:2369] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:01:27.804527 24744 tablet.cc:2369] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:01:27.835165 25203 raft_consensus.cc:482] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:01:27.835551 25203 raft_consensus.cc:504] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } } peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } }
I20260808 03:01:27.837703 25203 leader_election.cc:288] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efae2f315c024dabb8fa0bc555cd3147 (127.0.205.129:44389), 4bedfcae066e4f84b7a9d9890beb7814 (127.0.205.131:40079)
I20260808 03:01:27.844611 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.838403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:54893 (local address 127.0.205.131:40079)
0808 03:01:27.838602 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.838607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.839125 (+ 518us) server_negotiation.cc:407] Connection header received
0808 03:01:27.839314 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.839320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.839397 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.839502 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:27.840103 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.840859 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.842838 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.843634 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.843842 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.843929 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:27.844211 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 03:01:27.844341 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 03:01:27.844803 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.838036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:40079 (local address 127.0.205.130:54893)
0808 03:01:27.838951 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:27.838983 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:27.839151 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:27.839524 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:27.839529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:27.839555 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:27.839948 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.839962 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.841031 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.841042 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.842671 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.842683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.843862 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.843868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.844039 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.844068 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 03:01:27.844184 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 03:01:27.844329 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":26,"thread_start_us":461,"threads_started":1}
I20260808 03:01:27.844841 25188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.838194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:55409 (local address 127.0.205.129:44389)
0808 03:01:27.838420 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.838426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.839198 (+ 772us) server_negotiation.cc:407] Connection header received
0808 03:01:27.839422 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.839426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.839488 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.839592 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:01:27.840238 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.841194 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.843120 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.843874 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.844066 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.844148 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:27.844456 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 03:01:27.844588 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:01:27.844976 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.838046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44389 (local address 127.0.205.130:55409)
0808 03:01:27.839095 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:27.839118 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 03:01:27.839257 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:01:27.839605 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:01:27.839613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:01:27.839649 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:01:27.840071 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.840079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.841330 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.841338 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.842952 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:01:27.842965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.844114 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.844121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:01:27.844287 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.844311 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 03:01:27.844428 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 03:01:27.844555 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":38,"thread_start_us":739,"threads_started":1}
I20260808 03:01:27.846136 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a698e40d1de45e18df3175ce15a0a98" candidate_uuid: "f9a6ae68adf04408b6b6528648c83295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bedfcae066e4f84b7a9d9890beb7814" is_pre_election: true
I20260808 03:01:27.846168 24663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a698e40d1de45e18df3175ce15a0a98" candidate_uuid: "f9a6ae68adf04408b6b6528648c83295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efae2f315c024dabb8fa0bc555cd3147" is_pre_election: true
I20260808 03:01:27.846671 25084 raft_consensus.cc:2454] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9a6ae68adf04408b6b6528648c83295 in term 0.
I20260808 03:01:27.846673 24663 raft_consensus.cc:2454] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9a6ae68adf04408b6b6528648c83295 in term 0.
I20260808 03:01:27.847416 24773 leader_election.cc:302] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bedfcae066e4f84b7a9d9890beb7814, f9a6ae68adf04408b6b6528648c83295; no voters:
I20260808 03:01:27.847837 25203 raft_consensus.cc:2781] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:01:27.848006 25203 raft_consensus.cc:482] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:01:27.848121 25203 raft_consensus.cc:3037] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:01:27.850392 25203 raft_consensus.cc:504] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } } peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } }
I20260808 03:01:27.850994 25203 leader_election.cc:288] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [CANDIDATE]: Term 1 election: Requested vote from peers efae2f315c024dabb8fa0bc555cd3147 (127.0.205.129:44389), 4bedfcae066e4f84b7a9d9890beb7814 (127.0.205.131:40079)
I20260808 03:01:27.851545 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a698e40d1de45e18df3175ce15a0a98" candidate_uuid: "f9a6ae68adf04408b6b6528648c83295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bedfcae066e4f84b7a9d9890beb7814"
I20260808 03:01:27.851545 24663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a698e40d1de45e18df3175ce15a0a98" candidate_uuid: "f9a6ae68adf04408b6b6528648c83295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efae2f315c024dabb8fa0bc555cd3147"
I20260808 03:01:27.851769 25084 raft_consensus.cc:3037] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:01:27.851776 24663 raft_consensus.cc:3037] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:01:27.855119 24663 raft_consensus.cc:2454] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9a6ae68adf04408b6b6528648c83295 in term 1.
I20260808 03:01:27.855119 25084 raft_consensus.cc:2454] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9a6ae68adf04408b6b6528648c83295 in term 1.
I20260808 03:01:27.855607 24774 leader_election.cc:302] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: efae2f315c024dabb8fa0bc555cd3147, f9a6ae68adf04408b6b6528648c83295; no voters:
I20260808 03:01:27.855965 25203 raft_consensus.cc:2781] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:01:27.856431 25203 raft_consensus.cc:686] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 1 LEADER]: Becoming Leader. State: Replica: f9a6ae68adf04408b6b6528648c83295, State: Running, Role: LEADER
I20260808 03:01:27.856954 25203 consensus_queue.cc:237] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } } peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } }
I20260808 03:01:27.862154 24450 catalog_manager.cc:5495] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9a6ae68adf04408b6b6528648c83295 (127.0.205.130). New cstate: current_term: 1 leader_uuid: "f9a6ae68adf04408b6b6528648c83295" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9a6ae68adf04408b6b6528648c83295" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 36127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 } health_report { overall_health: UNKNOWN } } }
I20260808 03:01:27.930954 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.925502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45890 (local address 127.0.205.130:36127)
0808 03:01:27.925713 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.925717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.925733 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:01:27.925828 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.925832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.925959 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.926092 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:01:27.926495 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.927238 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.928760 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.929151 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.929244 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.929750 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:01:27.930407 (+ 657us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:01:27.930529 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:27.930644 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 03:01:27.930717 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:01:27.939625 24663 raft_consensus.cc:1261] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147 [term 1 FOLLOWER]: Refusing update from remote peer f9a6ae68adf04408b6b6528648c83295: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:01:27.939625 25084 raft_consensus.cc:1261] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814 [term 1 FOLLOWER]: Refusing update from remote peer f9a6ae68adf04408b6b6528648c83295: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:01:27.940737 25208 consensus_queue.cc:1035] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bedfcae066e4f84b7a9d9890beb7814" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 40079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:01:27.941108 25203 consensus_queue.cc:1035] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efae2f315c024dabb8fa0bc555cd3147" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:01:27.969703 24450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:01:27.970049 24450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:01:27.974001 24450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Sending DeleteTablet for 3 replicas of tablet 6a698e40d1de45e18df3175ce15a0a98
I20260808 03:01:27.975385 24627 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a698e40d1de45e18df3175ce15a0a98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:01:27 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:47538
I20260808 03:01:27.975430 25048 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a698e40d1de45e18df3175ce15a0a98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:01:27 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:37928
I20260808 03:01:27.975878 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 24505
I20260808 03:01:27.976419 24849 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a698e40d1de45e18df3175ce15a0a98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:01:27 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:45874
I20260808 03:01:27.978348 25230 tablet_replica.cc:330] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: stopping tablet replica
I20260808 03:01:27.978925 25230 raft_consensus.cc:2229] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:01:27.980929 25230 raft_consensus.cc:2258] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:01:27.982038 25231 tablet_replica.cc:330] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: stopping tablet replica
I20260808 03:01:27.982702 25231 raft_consensus.cc:2229] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:01:27.984485 25231 raft_consensus.cc:2258] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:01:27.985312 25227 tablet_replica.cc:330] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: stopping tablet replica
I20260808 03:01:27.985713 25227 raft_consensus.cc:2229] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:01:27.986222 25227 raft_consensus.cc:2258] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:01:27.986734 25230 ts_tablet_manager.cc:1905] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:01:27.988014 25231 ts_tablet_manager.cc:1905] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:01:27.995057 25230 ts_tablet_manager.cc:1918] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:01:27.995304 25230 log.cc:1196] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/6a698e40d1de45e18df3175ce15a0a98
I20260808 03:01:27.996078 25230 ts_tablet_manager.cc:1939] T 6a698e40d1de45e18df3175ce15a0a98 P efae2f315c024dabb8fa0bc555cd3147: Deleting consensus metadata
I20260808 03:01:27.997689 25231 ts_tablet_manager.cc:1918] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:01:27.998368 24435 catalog_manager.cc:4841] TS efae2f315c024dabb8fa0bc555cd3147 (127.0.205.129:44389): tablet 6a698e40d1de45e18df3175ce15a0a98 (table test-table [id=92da85b26837420abb74fc5886eaa82f]) successfully deleted
I20260808 03:01:27.998852 25231 log.cc:1196] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/6a698e40d1de45e18df3175ce15a0a98
I20260808 03:01:27.999161 25227 ts_tablet_manager.cc:1905] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:01:27.999490 25231 ts_tablet_manager.cc:1939] T 6a698e40d1de45e18df3175ce15a0a98 P f9a6ae68adf04408b6b6528648c83295: Deleting consensus metadata
I20260808 03:01:28.001507 24435 catalog_manager.cc:4841] TS f9a6ae68adf04408b6b6528648c83295 (127.0.205.130:36127): tablet 6a698e40d1de45e18df3175ce15a0a98 (table test-table [id=92da85b26837420abb74fc5886eaa82f]) successfully deleted
I20260808 03:01:28.001632 25188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:27.977141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47544 (local address 127.0.205.129:44389)
0808 03:01:27.977351 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:27.977360 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:27.977397 (+ 37us) server_negotiation.cc:407] Connection header received
0808 03:01:27.977474 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:27.977483 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:27.977599 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:27.977701 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:27.979594 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.980567 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.982793 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:27.983325 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:27.983477 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:27.993131 (+ 9654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:27.993146 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:27.993162 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:27.993206 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:27.997144 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:27.998747 (+ 1603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:27.998759 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:27.998763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:27.998956 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:27.999732 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:27.999740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:27.999743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:28.000235 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:28.000575 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:28.000947 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 03:01:28.001185 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:01:28.002949 24735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:28.006877 25227 ts_tablet_manager.cc:1918] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:01:28.007211 25227 log.cc:1196] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/6a698e40d1de45e18df3175ce15a0a98
I20260808 03:01:28.007697 25227 ts_tablet_manager.cc:1939] T 6a698e40d1de45e18df3175ce15a0a98 P 4bedfcae066e4f84b7a9d9890beb7814: Deleting consensus metadata
I20260808 03:01:28.009701 24433 catalog_manager.cc:4841] TS 4bedfcae066e4f84b7a9d9890beb7814 (127.0.205.131:40079): tablet 6a698e40d1de45e18df3175ce15a0a98 (table test-table [id=92da85b26837420abb74fc5886eaa82f]) successfully deleted
I20260808 03:01:28.129001 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 24505
I20260808 03:01:28.153587 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 24748
I20260808 03:01:28.171425 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:28.154493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45898 (local address 127.0.205.130:36127)
0808 03:01:28.154705 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:28.154712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:28.154754 (+ 42us) server_negotiation.cc:407] Connection header received
0808 03:01:28.154961 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:28.154967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:28.155079 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:28.155199 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:28.156609 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:28.158058 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:28.159652 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:28.160418 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:28.160554 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:28.163580 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:28.163597 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:28.163617 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:28.163666 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:28.168046 (+ 4380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:28.168854 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:28.168864 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:28.168868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:28.169062 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:28.169530 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:28.169536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:28.169539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:28.170099 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:28.170583 (+ 484us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:28.170751 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 03:01:28.171002 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 03:01:28.172209 24942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:28.291342 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 24748
I20260808 03:01:28.323975 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 24953
I20260808 03:01:28.338304 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:28.324736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37934 (local address 127.0.205.131:40079)
0808 03:01:28.324941 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 03:01:28.324946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:01:28.324991 (+ 45us) server_negotiation.cc:407] Connection header received
0808 03:01:28.325143 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:01:28.325147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:01:28.325211 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:01:28.325320 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:01:28.326875 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:28.327601 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:28.328897 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:01:28.329284 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:01:28.329379 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:01:28.332258 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:01:28.332273 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:01:28.332290 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:01:28.332323 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:01:28.335348 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:28.336235 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:28.336243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:28.336245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:28.336414 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:01:28.336773 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:01:28.336776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:01:28.336778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:01:28.337137 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:01:28.337366 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 03:01:28.337668 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 03:01:28.337843 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 03:01:28.339092 25152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:28.431604 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 24953
I20260808 03:01:28.455678 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 24403
I20260808 03:01:28.456924 24494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:01:28.574479 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 24403
I20260808 03:01:48.606002 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:36925
--webserver_interface=127.0.205.190
--webserver_port=35623
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:36925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:01:48.972196 25238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:48.972627 25238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:48.972810 25238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:48.980373 25238 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 03:01:48.980547 25238 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 03:01:48.980623 25238 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 03:01:48.980705 25238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:01:48.980815 25238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:48.980927 25238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:48.981024 25238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:01:48.981107 25238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:01:48.991457 25238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:36925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:36925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=35623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:48.992935 25238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:48.994380 25238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:49.006950 25244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:49.006886 25252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:49.006950 25247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:49.008066 25238 server_base.cc:1034] running on GCE node
I20260808 03:01:49.009009 25238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:49.010756 25238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:49.012022 25238 hybrid_clock.cc:648] HybridClock initialized: now 1786158109011957 us; error 50 us; skew 500 ppm
I20260808 03:01:49.012440 25238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:49 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158109, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:49.021759 25238 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:01:49.023582 25238 webserver.cc:460] Webserver started at http://127.0.205.190:35623/ using document root <none> and password file <none>
I20260808 03:01:49.024430 25238 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:49.024582 25238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:49.030822 25238 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 03:01:49.034108 25263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:49.035730 25238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:01:49.035979 25238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "ada042949a114aef932790d2e07a68a1"
format_stamp: "Formatted at 2026-08-08 03:01:02 on dist-test-slave-6ntk"
server_key: "095c939b18ae905754a6f75a181a093d"
server_key_iv: "6ccb5f22f385859f5816602c385cf3f7"
server_key_version: "encryptionkey@0"
I20260808 03:01:49.037058 25238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:49.053177 25238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:49.057708 25238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:49.058079 25238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:49.078244 25238 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:36925
I20260808 03:01:49.078265 25346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:36925 every 8 connection(s)
I20260808 03:01:49.079916 25238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:01:49.087224 25348 sys_catalog.cc:263] Verifying existing consensus state
I20260808 03:01:49.089658 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 25238
I20260808 03:01:49.090562 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:44389
--local_ip_for_outbound_sockets=127.0.205.129
--tserver_master_addrs=127.0.205.190:36925
--webserver_port=42981
--webserver_interface=127.0.205.129
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:01:49.090855 25348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Bootstrap starting.
I20260808 03:01:49.112174 25348 log.cc:824] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Log is configured to *not* fsync() on all Append() calls
I20260808 03:01:49.129297 25348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 03:01:49.130017 25348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Bootstrap complete.
I20260808 03:01:49.139667 25348 raft_consensus.cc:348] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:49.140672 25348 raft_consensus.cc:729] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada042949a114aef932790d2e07a68a1, State: Initialized, Role: FOLLOWER
I20260808 03:01:49.141438 25348 consensus_queue.cc:260] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:49.141698 25348 raft_consensus.cc:388] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:01:49.141851 25348 raft_consensus.cc:482] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:01:49.142118 25348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 2 FOLLOWER]: Advancing to term 3
I20260808 03:01:49.146360 25348 raft_consensus.cc:504] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:49.146929 25348 leader_election.cc:302] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ada042949a114aef932790d2e07a68a1; no voters:
I20260808 03:01:49.147645 25348 leader_election.cc:288] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 03:01:49.147794 25357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 03:01:49.149166 25357 raft_consensus.cc:686] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [term 3 LEADER]: Becoming Leader. State: Replica: ada042949a114aef932790d2e07a68a1, State: Running, Role: LEADER
I20260808 03:01:49.149905 25357 consensus_queue.cc:237] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } }
I20260808 03:01:49.150185 25348 sys_catalog.cc:564] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:01:49.153065 25360 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ada042949a114aef932790d2e07a68a1. Latest consensus state: current_term: 3 leader_uuid: "ada042949a114aef932790d2e07a68a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } } }
I20260808 03:01:49.153107 25359 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ada042949a114aef932790d2e07a68a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada042949a114aef932790d2e07a68a1" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36925 } } }
I20260808 03:01:49.153508 25360 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:01:49.153544 25359 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1 [sys.catalog]: This master's current role is: LEADER
I20260808 03:01:49.164882 25370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:01:49.176162 25370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:01:49.178275 25370 catalog_manager.cc:1259] Loaded cluster ID: 68b418f9dfb043c2bb0de8afcf8b0a49
I20260808 03:01:49.178485 25370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:01:49.184083 25370 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:01:49.186848 25370 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Loaded TSK: 0
I20260808 03:01:49.187353 25370 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ada042949a114aef932790d2e07a68a1: Loaded TSK: 1
F20260808 03:01:49.191722 25370 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f952368d9ad google::LogMessage::Fail() at ??:0
@ 0x7f952369088a google::LogMessage::SendToLog() at ??:0
@ 0x7f952368d4de google::LogMessage::Flush() at ??:0
@ 0x7f952368ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f95273f17d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f95273e073e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f95390bc1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f95390bb783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f95391472a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f95391470c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f95391915e9 std::function<>::operator()() at ??:0
@ 0x7f95390bfcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f95390bdc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9539145610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f9539145439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f95262d9345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f95262dc91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f95262dc799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f95262b7eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f95295916db start_thread at ??:0
@ 0x7f952252761f clone at ??:0
W20260808 03:01:49.510850 25352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:49.511162 25352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:49.511334 25352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:49.519065 25352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:49.519234 25352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:49.519443 25352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:01:49.529945 25352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:44389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=42981
--tserver_master_addrs=127.0.205.190:36925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:49.531422 25352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:49.532881 25352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:49.547111 25429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:49.547381 25430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:49.547420 25437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:49.547997 25352 server_base.cc:1034] running on GCE node
I20260808 03:01:49.548604 25352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:49.549599 25352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:49.550806 25352 hybrid_clock.cc:648] HybridClock initialized: now 1786158109550757 us; error 44 us; skew 500 ppm
I20260808 03:01:49.551167 25352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:49 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158109, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:49.554801 25352 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:01:49.556602 25352 webserver.cc:460] Webserver started at http://127.0.205.129:42981/ using document root <none> and password file <none>
I20260808 03:01:49.557406 25352 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:49.557557 25352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:49.563804 25352 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 03:01:49.566946 25451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:49.568468 25352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:01:49.568675 25352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "efae2f315c024dabb8fa0bc555cd3147"
format_stamp: "Formatted at 2026-08-08 03:01:03 on dist-test-slave-6ntk"
server_key: "8c76dbe72c9198b0feff4d914e742061"
server_key_iv: "c63174d185cde5643b18801793baa992"
server_key_version: "encryptionkey@0"
I20260808 03:01:49.569736 25352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:49.585248 25352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:49.589721 25352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:49.590056 25352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:49.591084 25352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:01:49.592464 25352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:01:49.592597 25352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:49.592756 25352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:01:49.592851 25352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:49.629135 25352 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:44389
I20260808 03:01:49.629232 25629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:44389 every 8 connection(s)
I20260808 03:01:49.630817 25352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:01:49.636253 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:49.634238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:46717)
0808 03:01:49.635032 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:49.635299 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":311,"threads_started":1}
W20260808 03:01:49.637009 25630 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:36925 (0 consecutive failures): Network error: Failed to ping master at 127.0.205.190:36925: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
I20260808 03:01:49.637614 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 25352
I20260808 03:01:49.638119 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:49.637660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:39609)
0808 03:01:49.637923 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:49.638005 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260808 03:01:49.638445 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:36127
--local_ip_for_outbound_sockets=127.0.205.130
--tserver_master_addrs=127.0.205.190:36925
--webserver_port=44889
--webserver_interface=127.0.205.130
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:01:49.639078 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:49.638717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:41131)
0808 03:01:49.638865 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:49.638954 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 03:01:49.639326 25630 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 03:01:50.000129 25633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:50.000443 25633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:50.000542 25633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:50.007997 25633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:50.008129 25633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:50.008266 25633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:01:50.018779 25633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:36127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=44889
--tserver_master_addrs=127.0.205.190:36925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:50.020220 25633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:50.021515 25633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:50.035616 25643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:50.035698 25642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:50.035672 25645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:50.036569 25633 server_base.cc:1034] running on GCE node
I20260808 03:01:50.037253 25633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:50.038442 25633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:50.039680 25633 hybrid_clock.cc:648] HybridClock initialized: now 1786158110039615 us; error 63 us; skew 500 ppm
I20260808 03:01:50.040020 25633 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:50 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:50.043540 25633 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:01:50.045173 25633 webserver.cc:460] Webserver started at http://127.0.205.130:44889/ using document root <none> and password file <none>
I20260808 03:01:50.045989 25633 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:50.046113 25633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:50.052620 25633 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 03:01:50.055599 25654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:50.057111 25633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:01:50.057370 25633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "f9a6ae68adf04408b6b6528648c83295"
format_stamp: "Formatted at 2026-08-08 03:01:03 on dist-test-slave-6ntk"
server_key: "077baba89221530f917432ccca35017d"
server_key_iv: "8eedfe2c5156cd9ecaa0785ef300b16b"
server_key_version: "encryptionkey@0"
I20260808 03:01:50.058597 25633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:50.088172 25633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:50.093032 25633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:50.093400 25633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:50.094573 25633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:01:50.096081 25633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:01:50.096174 25633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:50.096361 25633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:01:50.096450 25633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:50.133291 25633 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:36127
I20260808 03:01:50.133601 25813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:36127 every 8 connection(s)
I20260808 03:01:50.134922 25633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:01:50.140381 25816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:50.138237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:36269)
0808 03:01:50.139037 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:50.139393 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":304,"threads_started":1}
W20260808 03:01:50.141135 25814 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:36925 (0 consecutive failures): Network error: Failed to ping master at 127.0.205.190:36925: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
I20260808 03:01:50.142215 25816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:50.141808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:47787)
0808 03:01:50.142017 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:50.142094 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 03:01:50.143093 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 25633
I20260808 03:01:50.143129 25816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:50.142780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:41397)
0808 03:01:50.142933 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:50.143021 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 03:01:50.143472 25814 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 03:01:50.143857 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:40079
--local_ip_for_outbound_sockets=127.0.205.131
--tserver_master_addrs=127.0.205.190:36925
--webserver_port=42769
--webserver_interface=127.0.205.131
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:01:50.503805 25817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:01:50.504107 25817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:01:50.504279 25817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:01:50.511687 25817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:01:50.511824 25817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:01:50.512041 25817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:01:50.522298 25817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:40079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=42769
--tserver_master_addrs=127.0.205.190:36925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:01:50.523756 25817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:01:50.525218 25817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:01:50.539589 25822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:50.539610 25823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:01:50.539587 25827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:01:50.540745 25817 server_base.cc:1034] running on GCE node
I20260808 03:01:50.541468 25817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:01:50.542578 25817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:01:50.543807 25817 hybrid_clock.cc:648] HybridClock initialized: now 1786158110543736 us; error 77 us; skew 500 ppm
I20260808 03:01:50.544102 25817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:01:50 dist-test-slave-6ntk krb5kdc[23547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:01:50.547569 25817 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:01:50.549196 25817 webserver.cc:460] Webserver started at http://127.0.205.131:42769/ using document root <none> and password file <none>
I20260808 03:01:50.550019 25817 fs_manager.cc:359] Metadata directory not provided
I20260808 03:01:50.550179 25817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:01:50.556381 25817 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 03:01:50.559720 25835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:50.561262 25817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:01:50.561478 25817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "4bedfcae066e4f84b7a9d9890beb7814"
format_stamp: "Formatted at 2026-08-08 03:01:04 on dist-test-slave-6ntk"
server_key: "3aafa38080e1250662ee6d80701c5329"
server_key_iv: "f71fda32211a9ce0d9855fe347aed333"
server_key_version: "encryptionkey@0"
I20260808 03:01:50.562628 25817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:01:50.584069 25817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:01:50.588575 25817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:01:50.588881 25817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:01:50.590008 25817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:01:50.591405 25817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:01:50.591529 25817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:50.591666 25817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:01:50.591753 25817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:01:50.626559 25817 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:40079
I20260808 03:01:50.626622 25999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:40079 every 8 connection(s)
I20260808 03:01:50.628357 25817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:01:50.634137 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:50.631754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:57479)
0808 03:01:50.632615 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:50.632897 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":329,"threads_started":1}
W20260808 03:01:50.634891 26001 heartbeater.cc:644] Failed to heartbeat to 127.0.205.190:36925 (0 consecutive failures): Network error: Failed to ping master at 127.0.205.190:36925: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
I20260808 03:01:50.635802 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:50.635467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:36827)
0808 03:01:50.635618 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:50.635692 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 03:01:50.636689 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:50.636325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:35831)
0808 03:01:50.636448 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:50.636521 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 03:01:50.636960 26001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 03:01:50.638682 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 25817
I20260808 03:01:50.640834 26006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:50.639918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:39481)
0808 03:01:50.640474 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:50.640607 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":263,"threads_started":1}
I20260808 03:01:51.144950 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:51.144076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:50077)
0808 03:01:51.144605 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:51.144697 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":248,"threads_started":1}
I20260808 03:01:51.638525 26008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:51.637591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:50341)
0808 03:01:51.638155 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:51.638275 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":318,"threads_started":1}
I20260808 03:01:51.642700 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:51.641824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:57567)
0808 03:01:51.642439 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:51.642529 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":261,"threads_started":1}
I20260808 03:01:52.146958 26012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:52.146148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:53549)
0808 03:01:52.146679 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:52.146774 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":289,"threads_started":1}
I20260808 03:01:52.640275 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:52.639523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:44725)
0808 03:01:52.640012 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:52.640101 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":223,"threads_started":1}
I20260808 03:01:52.644385 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:52.643560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:53179)
0808 03:01:52.644115 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:52.644199 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":313,"threads_started":1}
I20260808 03:01:53.148658 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:53.147869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:49259)
0808 03:01:53.148331 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:53.148442 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":231,"threads_started":1}
I20260808 03:01:53.642174 26018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:53.641293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:42637)
0808 03:01:53.641810 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:53.641991 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":268,"threads_started":1}
I20260808 03:01:53.646251 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:53.645384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:38975)
0808 03:01:53.645832 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:53.645984 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":203,"threads_started":1}
I20260808 03:01:54.150480 26021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:54.149707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:53983)
0808 03:01:54.150207 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:54.150295 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":265,"threads_started":1}
I20260808 03:01:54.643989 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:54.643261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:43089)
0808 03:01:54.643733 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:54.643814 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":243,"threads_started":1}
I20260808 03:01:54.648052 26023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:54.647195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:43433)
0808 03:01:54.647777 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:54.647863 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":329,"threads_started":1}
I20260808 03:01:55.152099 26024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:55.151384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:47911)
0808 03:01:55.151837 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:55.151927 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":225,"threads_started":1}
I20260808 03:01:55.645862 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:55.645043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:41273)
0808 03:01:55.645561 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:55.645653 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":269,"threads_started":1}
I20260808 03:01:55.649812 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:55.648958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:57911)
0808 03:01:55.649539 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:55.649633 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":365,"threads_started":1}
I20260808 03:01:56.154026 26029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:56.153101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:46763)
0808 03:01:56.153668 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:56.153764 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":342,"threads_started":1}
I20260808 03:01:56.647850 26030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:56.646974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:58973)
0808 03:01:56.647593 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:56.647686 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":326,"threads_started":1}
I20260808 03:01:56.651578 26031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:56.650740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:45147)
0808 03:01:56.651283 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:56.651364 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":270,"threads_started":1}
I20260808 03:01:57.156286 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:57.155171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:50215)
0808 03:01:57.155905 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:57.156007 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":279,"threads_started":1}
I20260808 03:01:57.649705 26034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:57.648861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:54869)
0808 03:01:57.649394 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:57.649488 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":236,"threads_started":1}
I20260808 03:01:57.653479 26035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:57.652571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:52879)
0808 03:01:57.653149 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:57.653238 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":337,"threads_started":1}
I20260808 03:01:58.158264 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:58.157411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:33711)
0808 03:01:58.158006 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:58.158095 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":257,"threads_started":1}
I20260808 03:01:58.651414 26039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:58.650732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:48855)
0808 03:01:58.651156 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:58.651238 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":219,"threads_started":1}
I20260808 03:01:58.655120 26041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:58.654402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:36693)
0808 03:01:58.654852 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:58.654954 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":212,"threads_started":1}
I20260808 03:01:59.160204 26044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:59.159412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:42399)
0808 03:01:59.159930 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:59.160022 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":230,"threads_started":1}
I20260808 03:01:59.653205 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:59.652367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:57401)
0808 03:01:59.652903 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:59.652996 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":259,"threads_started":1}
I20260808 03:01:59.656875 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:01:59.656109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:52831)
0808 03:01:59.656639 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 03:01:59.656719 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":311,"threads_started":1}
I20260808 03:02:00.162089 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:00.161238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:53217)
0808 03:02:00.161689 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:00.161852 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":235,"threads_started":1}
I20260808 03:02:00.655160 26049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:00.654321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:40379)
0808 03:02:00.654883 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:00.654991 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":329,"threads_started":1}
I20260808 03:02:00.658649 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:00.657771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:52909)
0808 03:02:00.658332 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:00.658470 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":247,"threads_started":1}
I20260808 03:02:01.163786 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:01.162992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:51621)
0808 03:02:01.163478 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:01.163596 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":234,"threads_started":1}
I20260808 03:02:01.657027 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:01.656225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:58651)
0808 03:02:01.656740 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:01.656837 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":279,"threads_started":1}
I20260808 03:02:01.660334 26055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:01.659620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:58513)
0808 03:02:01.660108 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:01.660183 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":266,"threads_started":1}
I20260808 03:02:02.165666 26056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:02.164759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:36989)
0808 03:02:02.165282 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:02.165433 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":296,"threads_started":1}
I20260808 03:02:02.659006 26057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:02.658118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:34229)
0808 03:02:02.658747 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:02.658841 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":325,"threads_started":1}
I20260808 03:02:02.662272 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:02.661353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:48917)
0808 03:02:02.661981 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:02.662081 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":310,"threads_started":1}
I20260808 03:02:03.167939 26061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:03.166720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:33135)
0808 03:02:03.167482 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:03.167628 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":444,"threads_started":1}
I20260808 03:02:03.661562 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:03.660065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:42651)
0808 03:02:03.661126 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:03.661271 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":252,"threads_started":1}
I20260808 03:02:03.666591 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:03.663242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:50059)
0808 03:02:03.666250 (+ 3008us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:03.666332 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2906,"thread_start_us":2801,"threads_started":1}
I20260808 03:02:04.169690 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:04.168902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:58223)
0808 03:02:04.169393 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:04.169500 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":229,"threads_started":1}
I20260808 03:02:04.663519 26067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:04.662627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:60843)
0808 03:02:04.663218 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:04.663308 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":369,"threads_started":1}
I20260808 03:02:04.668697 26068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:04.667845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:51547)
0808 03:02:04.668419 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:04.668511 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":335,"threads_started":1}
I20260808 03:02:05.171749 26069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:05.170939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.130:47057)
0808 03:02:05.171446 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:05.171563 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":239,"threads_started":1}
I20260808 03:02:05.644706 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 25352
I20260808 03:02:05.664003 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:05.646064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45848 (local address 127.0.205.129:44389)
0808 03:02:05.646735 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:05.646748 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:05.646789 (+ 41us) server_negotiation.cc:407] Connection header received
0808 03:02:05.648420 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:05.648430 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:05.648868 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:05.649172 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:05.650838 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:05.652327 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:05.653762 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:05.654565 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:05.654736 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:05.657572 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:05.657587 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:05.657605 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:05.657644 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:05.660644 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:05.661423 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:05.661429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:05.661437 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:05.661608 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:05.662012 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:05.662016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:05.662018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:05.662662 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:05.662932 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:05.663296 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 03:02:05.663695 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":356,"threads_started":1}
I20260808 03:02:05.665640 25624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:05.665647 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:05.664750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.131:49451)
0808 03:02:05.665376 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:05.665479 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":350,"threads_started":1}
I20260808 03:02:05.755229 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:05.750310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36925 (local address 127.0.205.129:35035)
0808 03:02:05.754725 (+ 4415us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:05.754931 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.190:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4286,"thread_start_us":4135,"threads_started":1}
I20260808 03:02:05.755856 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 25352
I20260808 03:02:05.785111 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 25633
I20260808 03:02:05.804299 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:05.786174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.0.205.130:36127)
0808 03:02:05.786772 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:05.786787 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:05.786848 (+ 61us) server_negotiation.cc:407] Connection header received
0808 03:02:05.788492 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:05.788504 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:05.788961 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:05.789350 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:05.791017 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:05.792288 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:05.793625 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:05.794436 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:05.794662 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:05.797487 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:05.797495 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:05.797512 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:05.797554 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:05.800636 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:05.801411 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:05.801420 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:05.801450 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:05.801676 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:05.802160 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:05.802165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:05.802168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:05.802724 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:05.803092 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:05.803552 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 03:02:05.804002 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":301,"threads_started":1}
I20260808 03:02:05.805763 25808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:05.891233 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 25633
I20260808 03:02:05.919770 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 25817
I20260808 03:02:05.939007 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:05.921087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.0.205.131:40079)
0808 03:02:05.921728 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:05.921741 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:05.921787 (+ 46us) server_negotiation.cc:407] Connection header received
0808 03:02:05.923550 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:05.923560 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:05.924016 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:05.924309 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:05.925749 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:05.927218 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:05.928829 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:05.929514 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:05.929654 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:05.932602 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:05.932610 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:05.932627 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:05.932668 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:05.935854 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:05.936573 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:05.936580 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:05.936587 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:05.936757 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:05.937110 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:05.937113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:05.937115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:05.937641 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:05.937856 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:05.938343 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 03:02:05.938679 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":300,"threads_started":1}
I20260808 03:02:05.940462 25993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:06.025187 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 25817
2026-08-08T03:02:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65456 ms)
[----------] 30 tests from SecurityITest (314336 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:06 dist-test-slave-6ntk krb5kdc[26083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:06 dist-test-slave-6ntk krb5kdc[26083](info): set up 2 sockets
Aug 08 03:02:06 dist-test-slave-6ntk krb5kdc[26083](info): commencing operation
krb5kdc: starting...
W20260808 03:02:07.382537 822 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:07 dist-test-slave-6ntk krb5kdc[26083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:02:07.714681 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:36295
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:36295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:08.074776 26100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:08.075171 26100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:08.075336 26100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:08.082862 26100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:08.083053 26100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:08.083178 26100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:08.083282 26100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:08.083372 26100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:08.093636 26100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:36295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:36295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:08.095207 26100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:08.096715 26100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:08.109972 26110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:08.109988 26109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:08.109989 26112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:08.111192 26100 server_base.cc:1034] running on GCE node
I20260808 03:02:08.112082 26100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:08.113730 26100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:08.114969 26100 hybrid_clock.cc:648] HybridClock initialized: now 1786158128114903 us; error 59 us; skew 500 ppm
I20260808 03:02:08.115377 26100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:08 dist-test-slave-6ntk krb5kdc[26083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158128, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:08.118945 26100 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:02:08.120644 26100 webserver.cc:460] Webserver started at http://127.0.205.190:36597/ using document root <none> and password file <none>
I20260808 03:02:08.121397 26100 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:08.121551 26100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:08.121949 26100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:08.125104 26100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "33ca636b1d674b3bb9df549a7b524e41"
format_stamp: "Formatted at 2026-08-08 03:02:08 on dist-test-slave-6ntk"
server_key: "35df18ca80193e38dddf0f8ab12be774"
server_key_iv: "3f8c748a9043762429465973ce86ea8b"
server_key_version: "encryptionkey@0"
I20260808 03:02:08.126019 26100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "33ca636b1d674b3bb9df549a7b524e41"
format_stamp: "Formatted at 2026-08-08 03:02:08 on dist-test-slave-6ntk"
server_key: "35df18ca80193e38dddf0f8ab12be774"
server_key_iv: "3f8c748a9043762429465973ce86ea8b"
server_key_version: "encryptionkey@0"
I20260808 03:02:08.131529 26100 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 03:02:08.135260 26126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:08.136639 26100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 03:02:08.136847 26100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "33ca636b1d674b3bb9df549a7b524e41"
format_stamp: "Formatted at 2026-08-08 03:02:08 on dist-test-slave-6ntk"
server_key: "35df18ca80193e38dddf0f8ab12be774"
server_key_iv: "3f8c748a9043762429465973ce86ea8b"
server_key_version: "encryptionkey@0"
I20260808 03:02:08.137075 26100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:08.185957 26100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:08.192616 26100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:08.192881 26100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:08.212724 26100 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:36295
I20260808 03:02:08.212744 26202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:36295 every 8 connection(s)
I20260808 03:02:08.214377 26100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:08.218108 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 26100
I20260808 03:02:08.218353 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:08.218962 26203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:08.219148 822 external_mini_cluster.cc:1428] Setting key 1ff532e0aa331412f7f525a09b01cd5e
I20260808 03:02:08.235121 26203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41: Bootstrap starting.
Aug 08 03:02:08 dist-test-slave-6ntk krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:08.238700 26203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:08.239902 26203 log.cc:824] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:08.243294 26203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41: No bootstrap required, opened a new log
I20260808 03:02:08.244977 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:08.221296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.0.205.190:36295)
0808 03:02:08.222193 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:08.222207 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:08.222253 (+ 46us) server_negotiation.cc:407] Connection header received
0808 03:02:08.224053 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:08.224077 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:08.224493 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:08.224826 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:08.226868 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:08.228243 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:08.230326 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:08.231057 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:08.231205 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:08.236658 (+ 5453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:08.236668 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:08.236686 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:08.236730 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:08.240236 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:08.241317 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:08.241323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:08.241329 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:08.241574 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:08.242101 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:08.242114 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:08.242116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:08.242659 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:08.242999 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:08.243494 (+ 495us) server_negotiation.cc:299] Negotiation successful
0808 03:02:08.243953 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":398,"threads_started":1}
I20260808 03:02:08.252080 26203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca636b1d674b3bb9df549a7b524e41" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36295 } }
I20260808 03:02:08.252535 26203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:08.252673 26203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ca636b1d674b3bb9df549a7b524e41, State: Initialized, Role: FOLLOWER
I20260808 03:02:08.253314 26203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca636b1d674b3bb9df549a7b524e41" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36295 } }
I20260808 03:02:08.253543 26203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:08.253644 26203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:08.253782 26203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:08.256146 26203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca636b1d674b3bb9df549a7b524e41" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36295 } }
I20260808 03:02:08.256711 26203 leader_election.cc:302] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33ca636b1d674b3bb9df549a7b524e41; no voters:
I20260808 03:02:08.257501 26203 leader_election.cc:288] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:08.257664 26209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:08.258704 26209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [term 1 LEADER]: Becoming Leader. State: Replica: 33ca636b1d674b3bb9df549a7b524e41, State: Running, Role: LEADER
I20260808 03:02:08.259333 26209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca636b1d674b3bb9df549a7b524e41" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36295 } }
I20260808 03:02:08.260188 26203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:08.262583 26211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33ca636b1d674b3bb9df549a7b524e41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca636b1d674b3bb9df549a7b524e41" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36295 } } }
I20260808 03:02:08.262995 26211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:08.263458 26212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33ca636b1d674b3bb9df549a7b524e41. Latest consensus state: current_term: 1 leader_uuid: "33ca636b1d674b3bb9df549a7b524e41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ca636b1d674b3bb9df549a7b524e41" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 36295 } } }
I20260808 03:02:08.263782 26212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:08.266126 26217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:08.271731 26217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:08.281575 26217 catalog_manager.cc:1347] Generated new cluster ID: f6b09fa5905048809c8adeb109db2a35
I20260808 03:02:08.281708 26217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:08.309132 26217 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:08.310765 26217 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:08.324908 26217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41: Generated new TSK 0
I20260808 03:02:08.325974 26217 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 03:02:08.410897 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36295
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:08.779676 26238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:08.779979 26238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:08.780074 26238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:08.787926 26238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:08.788060 26238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:08.788187 26238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:02:08.798283 26238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:08.799603 26238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:08.800922 26238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:08.815131 26244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:08.815109 26245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:08.815052 26247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:08.815945 26238 server_base.cc:1034] running on GCE node
I20260808 03:02:08.816625 26238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:08.817652 26238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:08.818889 26238 hybrid_clock.cc:648] HybridClock initialized: now 1786158128818823 us; error 59 us; skew 500 ppm
I20260808 03:02:08.819231 26238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:08 dist-test-slave-6ntk krb5kdc[26083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158128, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:08.822676 26238 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:02:08.824267 26238 webserver.cc:460] Webserver started at http://127.0.205.129:36509/ using document root <none> and password file <none>
I20260808 03:02:08.825143 26238 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:08.825299 26238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:08.825589 26238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:08.828739 26238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "078a4074c5ba457f882b89333a8db751"
format_stamp: "Formatted at 2026-08-08 03:02:08 on dist-test-slave-6ntk"
server_key: "eb4b66da40fcc5a011c4f44e7fd04bbb"
server_key_iv: "d3135a4293940239b550065b7e5ce530"
server_key_version: "encryptionkey@0"
I20260808 03:02:08.829552 26238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "078a4074c5ba457f882b89333a8db751"
format_stamp: "Formatted at 2026-08-08 03:02:08 on dist-test-slave-6ntk"
server_key: "eb4b66da40fcc5a011c4f44e7fd04bbb"
server_key_iv: "d3135a4293940239b550065b7e5ce530"
server_key_version: "encryptionkey@0"
I20260808 03:02:08.834924 26238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 03:02:08.838927 26257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:08.840600 26238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260808 03:02:08.840775 26238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "078a4074c5ba457f882b89333a8db751"
format_stamp: "Formatted at 2026-08-08 03:02:08 on dist-test-slave-6ntk"
server_key: "eb4b66da40fcc5a011c4f44e7fd04bbb"
server_key_iv: "d3135a4293940239b550065b7e5ce530"
server_key_version: "encryptionkey@0"
I20260808 03:02:08.840976 26238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:08.868360 26238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:08.874075 26238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:08.874473 26238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:08.875684 26238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:08.877358 26238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:08.877451 26238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:08.877666 26238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:08.877769 26238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:08.914835 26238 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:46733
I20260808 03:02:08.914888 26423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:46733 every 8 connection(s)
I20260808 03:02:08.916237 26238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:02:08.924069 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 26238
I20260808 03:02:08.924322 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 03:02:08.925006 822 external_mini_cluster.cc:1428] Setting key c1614cf06ad6ef8a3beede6455fa6191
Aug 08 03:02:08 dist-test-slave-6ntk krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158128, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:08.937425 26426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:08.919108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:53741 (local address 127.0.205.190:36295)
0808 03:02:08.919572 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:08.919577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:08.920373 (+ 796us) server_negotiation.cc:407] Connection header received
0808 03:02:08.922411 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:08.922416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:08.922523 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:08.922648 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:08.924228 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:08.925027 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:08.926947 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:08.927348 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:08.927485 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:08.931334 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:08.931341 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:08.931344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:08.931379 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:08.934082 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:08.934900 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:08.934905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:08.934908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:08.935033 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:08.935527 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:08.935531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:08.935533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:08.935833 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:08.936002 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:08.936787 (+ 785us) server_negotiation.cc:299] Negotiation successful
0808 03:02:08.937044 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":249,"threads_started":1}
I20260808 03:02:08.938566 26428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:08.919489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36295 (local address 127.0.205.129:53741)
0808 03:02:08.920192 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:08.920239 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:08.922123 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:08.922900 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:08.922908 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:08.923459 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:08.924044 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:08.924063 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:08.925195 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:08.925200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:08.926710 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:08.926729 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:08.927598 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:08.927606 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:08.927853 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:08.928625 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:02:08.928650 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:08.931076 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:08.934236 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:08.934247 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:08.934266 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:08.934735 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:08.935156 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:08.935162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:08.935165 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:08.935390 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:08.935982 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:08.935990 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:08.936373 (+ 383us) client_negotiation.cc:769] Sending connection context
0808 03:02:08.936660 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 03:02:08.937242 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":283,"threads_started":1}
I20260808 03:02:08.940303 26424 heartbeater.cc:347] Connected to a master server at 127.0.205.190:36295
I20260808 03:02:08.940726 26424 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:08.941617 26424 heartbeater.cc:511] Master 127.0.205.190:36295 requested a full tablet report, sending...
I20260808 03:02:08.944082 26147 ts_manager.cc:194] Registered new tserver with Master: 078a4074c5ba457f882b89333a8db751 (127.0.205.129:46733)
I20260808 03:02:08.945776 26147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:53741
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 03:02:08.997706 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36295
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:09.354413 26434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:09.354713 26434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:09.354871 26434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:09.362478 26434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:09.362602 26434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:09.362780 26434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:02:09.372947 26434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:09.374353 26434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:09.375680 26434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:09.389285 26443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:09.389492 26441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:09.389554 26446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:09.390125 26434 server_base.cc:1034] running on GCE node
I20260808 03:02:09.390839 26434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:09.391857 26434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:09.393108 26434 hybrid_clock.cc:648] HybridClock initialized: now 1786158129393050 us; error 50 us; skew 500 ppm
I20260808 03:02:09.393503 26434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:09 dist-test-slave-6ntk krb5kdc[26083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158129, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:09.396965 26434 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:02:09.398566 26434 webserver.cc:460] Webserver started at http://127.0.205.130:38315/ using document root <none> and password file <none>
I20260808 03:02:09.399240 26434 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:09.399355 26434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:09.399624 26434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:09.402669 26434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "5a48e098fa6345dfa1c170cb81fd9fb9"
format_stamp: "Formatted at 2026-08-08 03:02:09 on dist-test-slave-6ntk"
server_key: "9b2988b5cd2cc6a4379f6960abaa4a60"
server_key_iv: "1cc783525c4937632a83ad21a0ea7181"
server_key_version: "encryptionkey@0"
I20260808 03:02:09.403450 26434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "5a48e098fa6345dfa1c170cb81fd9fb9"
format_stamp: "Formatted at 2026-08-08 03:02:09 on dist-test-slave-6ntk"
server_key: "9b2988b5cd2cc6a4379f6960abaa4a60"
server_key_iv: "1cc783525c4937632a83ad21a0ea7181"
server_key_version: "encryptionkey@0"
I20260808 03:02:09.408883 26434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:02:09.412978 26455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:09.414655 26434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:02:09.414808 26434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "5a48e098fa6345dfa1c170cb81fd9fb9"
format_stamp: "Formatted at 2026-08-08 03:02:09 on dist-test-slave-6ntk"
server_key: "9b2988b5cd2cc6a4379f6960abaa4a60"
server_key_iv: "1cc783525c4937632a83ad21a0ea7181"
server_key_version: "encryptionkey@0"
I20260808 03:02:09.414984 26434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:09.440269 26434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:09.444721 26434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:09.445026 26434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:09.446107 26434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:09.447435 26434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:09.447553 26434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:09.447685 26434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:09.447749 26434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:09.480947 26434 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:42459
I20260808 03:02:09.480983 26614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:42459 every 8 connection(s)
I20260808 03:02:09.482318 26434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:02:09.490474 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 26434
I20260808 03:02:09.490686 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 03:02:09.491137 822 external_mini_cluster.cc:1428] Setting key b103a29fe706ec8e1db5434a8180604a
Aug 08 03:02:09 dist-test-slave-6ntk krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158129, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:09.505805 26620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:09.485176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:41007 (local address 127.0.205.190:36295)
0808 03:02:09.485646 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:09.485651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:09.486716 (+ 1065us) server_negotiation.cc:407] Connection header received
0808 03:02:09.488521 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:09.488525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:09.488604 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:09.488700 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:09.490524 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:09.491501 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:09.493529 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:09.494088 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:09.494262 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:09.498131 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:09.498137 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:09.498140 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:09.498177 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:09.501814 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:09.502868 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:09.502875 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:09.502881 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:09.503049 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:09.503568 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:09.503574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:09.503579 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:09.503959 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:09.504108 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:09.505051 (+ 943us) server_negotiation.cc:299] Negotiation successful
0808 03:02:09.505326 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":233,"threads_started":1}
I20260808 03:02:09.507215 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:09.485722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36295 (local address 127.0.205.130:41007)
0808 03:02:09.486537 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:09.486576 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:09.488251 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:09.488919 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:09.488927 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:09.489500 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:09.490320 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:09.490344 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:09.491662 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:09.491670 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:09.493375 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:09.493389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:09.494382 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:09.494391 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:09.494638 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:09.495429 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:02:09.495454 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:09.497818 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:09.502010 (+ 4192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:09.502022 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:09.502044 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:09.502728 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:09.503165 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:09.503171 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:09.503177 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:09.503445 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:09.504095 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:09.504107 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:09.504606 (+ 499us) client_negotiation.cc:769] Sending connection context
0808 03:02:09.504940 (+ 334us) client_negotiation.cc:240] Negotiation successful
0808 03:02:09.505568 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":362,"threads_started":1}
I20260808 03:02:09.508850 26617 heartbeater.cc:347] Connected to a master server at 127.0.205.190:36295
I20260808 03:02:09.509184 26617 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:09.510013 26617 heartbeater.cc:511] Master 127.0.205.190:36295 requested a full tablet report, sending...
I20260808 03:02:09.511639 26147 ts_manager.cc:194] Registered new tserver with Master: 5a48e098fa6345dfa1c170cb81fd9fb9 (127.0.205.130:42459)
I20260808 03:02:09.512465 26147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:41007
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 03:02:09.565855 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36295
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:42319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:09.928498 26625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:09.928901 26625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:09.929092 26625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:09.936987 26625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:09.937165 26625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:09.937357 26625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:02:09.948258 26625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:42319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:36295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:09.949141 26424 heartbeater.cc:503] Master 127.0.205.190:36295 was elected leader, sending a full tablet report...
I20260808 03:02:09.949849 26625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:09.951810 26625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:09.967456 26631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:09.967540 26633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:09.967422 26638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:09.971729 26625 server_base.cc:1034] running on GCE node
I20260808 03:02:09.972430 26625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:09.973554 26625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:09.974763 26625 hybrid_clock.cc:648] HybridClock initialized: now 1786158129974684 us; error 72 us; skew 500 ppm
I20260808 03:02:09.975109 26625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:09 dist-test-slave-6ntk krb5kdc[26083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158129, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:09.978591 26625 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:02:09.980307 26625 webserver.cc:460] Webserver started at http://127.0.205.131:35663/ using document root <none> and password file <none>
I20260808 03:02:09.981079 26625 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:09.981233 26625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:09.981573 26625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:09.984658 26625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "2cebb9d1e4364cce947cddf1254c12e8"
format_stamp: "Formatted at 2026-08-08 03:02:09 on dist-test-slave-6ntk"
server_key: "1a57dcdbf975218405cc2ec015b5e1a6"
server_key_iv: "698cd0f4ca7d768b9955d0497212f28b"
server_key_version: "encryptionkey@0"
I20260808 03:02:09.985479 26625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "2cebb9d1e4364cce947cddf1254c12e8"
format_stamp: "Formatted at 2026-08-08 03:02:09 on dist-test-slave-6ntk"
server_key: "1a57dcdbf975218405cc2ec015b5e1a6"
server_key_iv: "698cd0f4ca7d768b9955d0497212f28b"
server_key_version: "encryptionkey@0"
I20260808 03:02:09.990978 26625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 03:02:09.994609 26650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:09.995999 26625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:09.996186 26625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "2cebb9d1e4364cce947cddf1254c12e8"
format_stamp: "Formatted at 2026-08-08 03:02:09 on dist-test-slave-6ntk"
server_key: "1a57dcdbf975218405cc2ec015b5e1a6"
server_key_iv: "698cd0f4ca7d768b9955d0497212f28b"
server_key_version: "encryptionkey@0"
I20260808 03:02:09.996394 26625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:10.017607 26625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:10.022429 26625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:10.022745 26625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:10.023767 26625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:10.025138 26625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:10.025218 26625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:10.025365 26625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:10.025429 26625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:10.062441 26625 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:38811
I20260808 03:02:10.062505 26803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:38811 every 8 connection(s)
I20260808 03:02:10.063923 26625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:02:10.069020 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 26625
I20260808 03:02:10.069217 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 03:02:10.069751 822 external_mini_cluster.cc:1428] Setting key 307df6f1d35f0bae2fe604ea3f9fcb8c
Aug 08 03:02:10 dist-test-slave-6ntk krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158129, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:10.085068 26808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.066804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:50981 (local address 127.0.205.190:36295)
0808 03:02:10.067286 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.067291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.068253 (+ 962us) server_negotiation.cc:407] Connection header received
0808 03:02:10.070716 (+ 2463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.070722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.070843 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.070986 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:10.072749 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.073659 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.075291 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.075674 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.075769 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.079231 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:10.079236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:10.079240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:10.079273 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:10.081765 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:10.082519 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:10.082523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:10.082525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:10.082657 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:10.083046 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:10.083049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:10.083052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:10.083415 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:10.083572 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.084403 (+ 831us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.084655 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":231,"threads_started":1}
I20260808 03:02:10.086123 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.067357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:36295 (local address 127.0.205.131:50981)
0808 03:02:10.068085 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:10.068124 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:10.070407 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:10.071267 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:10.071282 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:10.071966 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:10.072558 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.072576 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.073830 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.073834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.075163 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.075175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.075884 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.075889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.076097 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.076740 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:02:10.076767 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:10.079021 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:10.081942 (+ 2921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:10.081949 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:10.081963 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:10.082398 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:10.082751 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:10.082754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:10.082756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:10.082947 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:10.083561 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:10.083569 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:10.083979 (+ 410us) client_negotiation.cc:769] Sending connection context
0808 03:02:10.084263 (+ 284us) client_negotiation.cc:240] Negotiation successful
0808 03:02:10.084770 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":275,"threads_started":1}
I20260808 03:02:10.087484 26804 heartbeater.cc:347] Connected to a master server at 127.0.205.190:36295
I20260808 03:02:10.087800 26804 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:10.088547 26804 heartbeater.cc:511] Master 127.0.205.190:36295 requested a full tablet report, sending...
I20260808 03:02:10.090137 26147 ts_manager.cc:194] Registered new tserver with Master: 2cebb9d1e4364cce947cddf1254c12e8 (127.0.205.131:38811)
I20260808 03:02:10.091089 26147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:50981
I20260808 03:02:10.098068 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:02:10.116276 26808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.102923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55288 (local address 127.0.205.190:36295)
0808 03:02:10.103119 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.103128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.103565 (+ 437us) server_negotiation.cc:407] Connection header received
0808 03:02:10.103771 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.103776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.103886 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.103994 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:10.105483 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.106295 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.107602 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.108049 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.108191 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.110692 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:10.110697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:10.110700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:10.110734 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:10.113344 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:10.114168 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:10.114175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:10.114178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:10.114307 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:10.114715 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:10.114720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:10.114724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:10.115088 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:10.115342 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.115710 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.115947 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:02:10.120512 26147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:02:10.127593 26147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:02:10.151911 26827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.141916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.0.205.131:38811)
0808 03:02:10.143036 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.143044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.143584 (+ 540us) server_negotiation.cc:407] Connection header received
0808 03:02:10.143885 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.143893 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.144111 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.144313 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:10.145451 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.147091 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.149667 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.150660 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.150806 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.151068 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.151359 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.151542 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":752,"threads_started":1}
I20260808 03:02:10.152253 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.140432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:42459 (local address 127.0.0.1:52404)
0808 03:02:10.142330 (+ 1898us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:10.142368 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:10.142543 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:10.143860 (+ 1317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:10.143867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:10.143897 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:10.144359 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.144377 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.146816 (+ 2439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.146826 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.149401 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.149418 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.151323 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.151329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.151485 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.151517 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 03:02:10.151601 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 03:02:10.151799 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1594,"mutex_wait_us":4,"thread_start_us":2739,"threads_started":1}
I20260808 03:02:10.152984 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.140670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52404 (local address 127.0.205.130:42459)
0808 03:02:10.143153 (+ 2483us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.143162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.143193 (+ 31us) server_negotiation.cc:407] Connection header received
0808 03:02:10.143303 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.143313 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.143607 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.144568 (+ 961us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:10.144895 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.146638 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.149597 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.151117 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.151264 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.151448 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.152474 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.152650 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2308,"thread_start_us":2216,"threads_started":1}
I20260808 03:02:10.153033 26824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.141157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:38811 (local address 127.0.0.1:56064)
0808 03:02:10.143479 (+ 2322us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:10.143496 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:10.143689 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:10.144678 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:10.144685 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:10.144712 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:10.145210 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.145224 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.147224 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.147228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.149539 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.149551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.150856 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.150862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.151061 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.151114 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 03:02:10.152203 (+ 1089us) client_negotiation.cc:240] Negotiation successful
0808 03:02:10.152633 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2215,"mutex_wait_us":51,"thread_start_us":749,"threads_started":1}
I20260808 03:02:10.153936 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.144272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:46733 (local address 127.0.0.1:56922)
0808 03:02:10.144913 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:10.144937 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:10.145091 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:10.146324 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:10.146331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:10.146358 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:10.146787 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.146801 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.148624 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.148632 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.150644 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.150659 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.152994 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.153000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.153214 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.153242 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 03:02:10.153325 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 03:02:10.153452 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":63,"thread_start_us":431,"threads_started":1}
I20260808 03:02:10.154628 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.143798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56922 (local address 127.0.205.129:46733)
0808 03:02:10.145733 (+ 1935us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.145757 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.145783 (+ 26us) server_negotiation.cc:407] Connection header received
0808 03:02:10.145936 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.145943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.146152 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.146349 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:10.146964 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.148481 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.150844 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.152512 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.152651 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.152883 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.154183 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.154389 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1746,"thread_start_us":1584,"threads_started":1}
I20260808 03:02:10.156999 26526 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea8c9f679ed4192b0a498e6e6740a4e (DEFAULT_TABLE table=test-table [id=4135d7e0d31141e5838d5f7c893b5c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:10.157001 26710 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea8c9f679ed4192b0a498e6e6740a4e (DEFAULT_TABLE table=test-table [id=4135d7e0d31141e5838d5f7c893b5c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:10.157708 26334 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea8c9f679ed4192b0a498e6e6740a4e (DEFAULT_TABLE table=test-table [id=4135d7e0d31141e5838d5f7c893b5c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:10.158558 26526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea8c9f679ed4192b0a498e6e6740a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:10.158557 26710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea8c9f679ed4192b0a498e6e6740a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:10.159087 26334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea8c9f679ed4192b0a498e6e6740a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:10.172382 26838 tablet_bootstrap.cc:492] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: Bootstrap starting.
I20260808 03:02:10.174688 26841 tablet_bootstrap.cc:492] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: Bootstrap starting.
I20260808 03:02:10.176990 26838 tablet_bootstrap.cc:654] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:10.178386 26841 tablet_bootstrap.cc:654] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:10.178413 26838 log.cc:824] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:10.178745 26842 tablet_bootstrap.cc:492] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: Bootstrap starting.
I20260808 03:02:10.179699 26841 log.cc:824] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:10.181936 26842 tablet_bootstrap.cc:654] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:10.183260 26842 log.cc:824] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:10.184785 26838 tablet_bootstrap.cc:492] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: No bootstrap required, opened a new log
I20260808 03:02:10.184902 26841 tablet_bootstrap.cc:492] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: No bootstrap required, opened a new log
I20260808 03:02:10.185123 26838 ts_tablet_manager.cc:1397] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260808 03:02:10.185181 26841 ts_tablet_manager.cc:1397] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260808 03:02:10.186451 26842 tablet_bootstrap.cc:492] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: No bootstrap required, opened a new log
I20260808 03:02:10.186698 26842 ts_tablet_manager.cc:1397] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260808 03:02:10.196424 26841 raft_consensus.cc:348] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 } } peers { permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 } } peers { permanent_uuid: "2cebb9d1e4364cce947cddf1254c12e8" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 38811 } }
I20260808 03:02:10.196516 26838 raft_consensus.cc:348] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 } } peers { permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 } } peers { permanent_uuid: "2cebb9d1e4364cce947cddf1254c12e8" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 38811 } }
I20260808 03:02:10.196946 26841 raft_consensus.cc:374] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:10.196975 26838 raft_consensus.cc:374] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:10.197057 26841 raft_consensus.cc:729] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cebb9d1e4364cce947cddf1254c12e8, State: Initialized, Role: FOLLOWER
I20260808 03:02:10.197080 26838 raft_consensus.cc:729] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 078a4074c5ba457f882b89333a8db751, State: Initialized, Role: FOLLOWER
I20260808 03:02:10.197258 26842 raft_consensus.cc:348] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 } } peers { permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 } } peers { permanent_uuid: "2cebb9d1e4364cce947cddf1254c12e8" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 38811 } }
I20260808 03:02:10.197835 26842 raft_consensus.cc:374] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:10.197726 26838 consensus_queue.cc:260] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 } } peers { permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 } } peers { permanent_uuid: "2cebb9d1e4364cce947cddf1254c12e8" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 38811 } }
I20260808 03:02:10.197960 26842 raft_consensus.cc:729] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a48e098fa6345dfa1c170cb81fd9fb9, State: Initialized, Role: FOLLOWER
I20260808 03:02:10.197820 26841 consensus_queue.cc:260] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 } } peers { permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 } } peers { permanent_uuid: "2cebb9d1e4364cce947cddf1254c12e8" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 38811 } }
I20260808 03:02:10.198796 26842 consensus_queue.cc:260] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 } } peers { permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 } } peers { permanent_uuid: "2cebb9d1e4364cce947cddf1254c12e8" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 38811 } }
I20260808 03:02:10.200256 26838 ts_tablet_manager.cc:1428] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: Time spent starting tablet: real 0.015s user 0.016s sys 0.001s
I20260808 03:02:10.203310 26617 heartbeater.cc:503] Master 127.0.205.190:36295 was elected leader, sending a full tablet report...
I20260808 03:02:10.204598 26842 ts_tablet_manager.cc:1428] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: Time spent starting tablet: real 0.018s user 0.014s sys 0.002s
I20260808 03:02:10.205621 26804 heartbeater.cc:503] Master 127.0.205.190:36295 was elected leader, sending a full tablet report...
I20260808 03:02:10.206354 26841 ts_tablet_manager.cc:1428] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: Time spent starting tablet: real 0.021s user 0.014s sys 0.009s
W20260808 03:02:10.234413 26619 tablet.cc:2369] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:02:10.315804 26805 tablet.cc:2369] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:02:10.417928 26847 raft_consensus.cc:482] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:02:10.418354 26847 raft_consensus.cc:504] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 } } peers { permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 } } peers { permanent_uuid: "2cebb9d1e4364cce947cddf1254c12e8" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 38811 } }
W20260808 03:02:10.419032 26425 tablet.cc:2369] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:02:10.420667 26847 leader_election.cc:288] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a48e098fa6345dfa1c170cb81fd9fb9 (127.0.205.130:42459), 078a4074c5ba457f882b89333a8db751 (127.0.205.129:46733)
I20260808 03:02:10.428974 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.421835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:34037 (local address 127.0.205.130:42459)
0808 03:02:10.422105 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.422112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.422142 (+ 30us) server_negotiation.cc:407] Connection header received
0808 03:02:10.422357 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.422362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.422456 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.422561 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:10.423205 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.424284 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.426766 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.428031 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.428176 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.428233 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.428648 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.428747 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 03:02:10.429072 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.421802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:42459 (local address 127.0.205.131:34037)
0808 03:02:10.422055 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:10.422073 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:10.422186 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:10.422618 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:10.422626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:10.422689 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:10.423008 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.423025 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.424432 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.424438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.426598 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.426611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.428259 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.428267 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.428465 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.428511 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 03:02:10.428624 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 03:02:10.428779 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":36}
I20260808 03:02:10.430284 26552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea8c9f679ed4192b0a498e6e6740a4e" candidate_uuid: "2cebb9d1e4364cce947cddf1254c12e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" is_pre_election: true
I20260808 03:02:10.430452 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.422624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:53687 (local address 127.0.205.129:46733)
0808 03:02:10.422823 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.422827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.423453 (+ 626us) server_negotiation.cc:407] Connection header received
0808 03:02:10.423604 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.423608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.423689 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.423772 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:10.424433 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.425531 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.428159 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.429466 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.429591 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.429641 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.430043 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.430175 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 03:02:10.430523 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.422320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:46733 (local address 127.0.205.131:53687)
0808 03:02:10.423338 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:10.423356 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:10.423494 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:10.423813 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:10.423818 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:10.423843 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:10.424209 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.424219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.425717 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.425721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.427919 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:10.427939 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.429655 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.429663 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:10.429833 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.429858 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 03:02:10.430012 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 03:02:10.430142 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"mutex_wait_us":86,"thread_start_us":642,"threads_started":1}
I20260808 03:02:10.430866 26552 raft_consensus.cc:2454] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cebb9d1e4364cce947cddf1254c12e8 in term 0.
I20260808 03:02:10.431639 26654 leader_election.cc:302] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cebb9d1e4364cce947cddf1254c12e8, 5a48e098fa6345dfa1c170cb81fd9fb9; no voters:
I20260808 03:02:10.431589 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea8c9f679ed4192b0a498e6e6740a4e" candidate_uuid: "2cebb9d1e4364cce947cddf1254c12e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "078a4074c5ba457f882b89333a8db751" is_pre_election: true
I20260808 03:02:10.432047 26847 raft_consensus.cc:2781] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:02:10.432224 26847 raft_consensus.cc:482] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:02:10.432154 26358 raft_consensus.cc:2454] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cebb9d1e4364cce947cddf1254c12e8 in term 0.
I20260808 03:02:10.432340 26847 raft_consensus.cc:3037] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:10.434717 26847 raft_consensus.cc:504] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 } } peers { permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 } } peers { permanent_uuid: "2cebb9d1e4364cce947cddf1254c12e8" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 38811 } }
I20260808 03:02:10.435250 26847 leader_election.cc:288] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [CANDIDATE]: Term 1 election: Requested vote from peers 5a48e098fa6345dfa1c170cb81fd9fb9 (127.0.205.130:42459), 078a4074c5ba457f882b89333a8db751 (127.0.205.129:46733)
I20260808 03:02:10.435745 26552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea8c9f679ed4192b0a498e6e6740a4e" candidate_uuid: "2cebb9d1e4364cce947cddf1254c12e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9"
I20260808 03:02:10.435813 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea8c9f679ed4192b0a498e6e6740a4e" candidate_uuid: "2cebb9d1e4364cce947cddf1254c12e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "078a4074c5ba457f882b89333a8db751"
I20260808 03:02:10.435950 26552 raft_consensus.cc:3037] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:10.436077 26358 raft_consensus.cc:3037] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:10.438289 26552 raft_consensus.cc:2454] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cebb9d1e4364cce947cddf1254c12e8 in term 1.
I20260808 03:02:10.438397 26358 raft_consensus.cc:2454] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cebb9d1e4364cce947cddf1254c12e8 in term 1.
I20260808 03:02:10.438726 26654 leader_election.cc:302] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cebb9d1e4364cce947cddf1254c12e8, 5a48e098fa6345dfa1c170cb81fd9fb9; no voters:
I20260808 03:02:10.439111 26847 raft_consensus.cc:2781] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:10.439604 26847 raft_consensus.cc:686] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 1 LEADER]: Becoming Leader. State: Replica: 2cebb9d1e4364cce947cddf1254c12e8, State: Running, Role: LEADER
I20260808 03:02:10.440218 26847 consensus_queue.cc:237] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 } } peers { permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 } } peers { permanent_uuid: "2cebb9d1e4364cce947cddf1254c12e8" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 38811 } }
I20260808 03:02:10.445362 26147 catalog_manager.cc:5495] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cebb9d1e4364cce947cddf1254c12e8 (127.0.205.131). New cstate: current_term: 1 leader_uuid: "2cebb9d1e4364cce947cddf1254c12e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cebb9d1e4364cce947cddf1254c12e8" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 38811 } health_report { overall_health: HEALTHY } } }
I20260808 03:02:10.527467 26827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.521313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56078 (local address 127.0.205.131:38811)
0808 03:02:10.521537 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.521544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.521566 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:02:10.521799 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.521803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.521981 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.522157 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:02:10.522592 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.523356 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.524901 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.525381 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.525539 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.526187 (+ 648us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:02:10.526742 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:02:10.526921 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.527041 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.527146 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 03:02:10.535777 26552 raft_consensus.cc:1261] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9 [term 1 FOLLOWER]: Refusing update from remote peer 2cebb9d1e4364cce947cddf1254c12e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:10.535777 26358 raft_consensus.cc:1261] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751 [term 1 FOLLOWER]: Refusing update from remote peer 2cebb9d1e4364cce947cddf1254c12e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:10.536830 26855 consensus_queue.cc:1035] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "078a4074c5ba457f882b89333a8db751" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 46733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:10.537308 26847 consensus_queue.cc:1035] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a48e098fa6345dfa1c170cb81fd9fb9" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:10.548556 26860 mvcc.cc:205] Tried to move back new op lower bound from 7316103702663880704 to 7316103702286921728. Current Snapshot: MvccSnapshot[applied={T|T < 7316103702663880704}]
I20260808 03:02:10.565237 26147 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:10.565609 26147 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:10.569737 26147 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33ca636b1d674b3bb9df549a7b524e41: Sending DeleteTablet for 3 replicas of tablet 2ea8c9f679ed4192b0a498e6e6740a4e
I20260808 03:02:10.571473 26334 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ea8c9f679ed4192b0a498e6e6740a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:10 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:56922
I20260808 03:02:10.571570 26710 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ea8c9f679ed4192b0a498e6e6740a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:10 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:56064
I20260808 03:02:10.572290 26526 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ea8c9f679ed4192b0a498e6e6740a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:10 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:52404
I20260808 03:02:10.574792 26873 tablet_replica.cc:330] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: stopping tablet replica
I20260808 03:02:10.574870 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 26238
I20260808 03:02:10.575758 26870 tablet_replica.cc:330] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: stopping tablet replica
I20260808 03:02:10.576284 26870 raft_consensus.cc:2229] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:10.576736 26871 tablet_replica.cc:330] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: stopping tablet replica
I20260808 03:02:10.576870 26870 raft_consensus.cc:2258] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:10.577805 26873 raft_consensus.cc:2229] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:10.578118 26871 raft_consensus.cc:2229] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:02:10.579051 26871 raft_consensus.cc:2258] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:10.579145 26870 ts_tablet_manager.cc:1905] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:10.579661 26873 raft_consensus.cc:2258] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:10.582163 26873 ts_tablet_manager.cc:1905] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:10.584280 26871 ts_tablet_manager.cc:1905] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:10.586782 26870 ts_tablet_manager.cc:1918] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:10.586968 26870 log.cc:1196] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/2ea8c9f679ed4192b0a498e6e6740a4e
I20260808 03:02:10.587558 26870 ts_tablet_manager.cc:1939] T 2ea8c9f679ed4192b0a498e6e6740a4e P 078a4074c5ba457f882b89333a8db751: Deleting consensus metadata
Aug 08 03:02:10 dist-test-slave-6ntk krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 03:02:10.589641 26131 catalog_manager.cc:4841] TS 078a4074c5ba457f882b89333a8db751 (127.0.205.129:46733): tablet 2ea8c9f679ed4192b0a498e6e6740a4e (table test-table [id=4135d7e0d31141e5838d5f7c893b5c06]) successfully deleted
I20260808 03:02:10.593848 26873 ts_tablet_manager.cc:1918] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:10.594152 26873 log.cc:1196] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/2ea8c9f679ed4192b0a498e6e6740a4e
I20260808 03:02:10.594759 26873 ts_tablet_manager.cc:1939] T 2ea8c9f679ed4192b0a498e6e6740a4e P 5a48e098fa6345dfa1c170cb81fd9fb9: Deleting consensus metadata
I20260808 03:02:10.596719 26131 catalog_manager.cc:4841] TS 5a48e098fa6345dfa1c170cb81fd9fb9 (127.0.205.130:42459): tablet 2ea8c9f679ed4192b0a498e6e6740a4e (table test-table [id=4135d7e0d31141e5838d5f7c893b5c06]) successfully deleted
I20260808 03:02:10.597911 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.576374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56938 (local address 127.0.205.129:46733)
0808 03:02:10.576617 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.576623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.577420 (+ 797us) server_negotiation.cc:407] Connection header received
0808 03:02:10.577629 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.577634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.577724 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.577842 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:10.579839 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.580991 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.582982 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.583678 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.583789 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.589638 (+ 5849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:10.589651 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:10.589678 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:10.589728 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:10.593842 (+ 4114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:10.594961 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:10.594973 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:10.594977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:10.595156 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:10.595654 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:10.595659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:10.595663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:10.596163 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:10.596481 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.597204 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.597491 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260808 03:02:10.599526 26418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:10.600258 26871 ts_tablet_manager.cc:1918] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:10.600575 26871 log.cc:1196] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/2ea8c9f679ed4192b0a498e6e6740a4e
I20260808 03:02:10.601863 26871 ts_tablet_manager.cc:1939] T 2ea8c9f679ed4192b0a498e6e6740a4e P 2cebb9d1e4364cce947cddf1254c12e8: Deleting consensus metadata
I20260808 03:02:10.604203 26128 catalog_manager.cc:4841] TS 2cebb9d1e4364cce947cddf1254c12e8 (127.0.205.131:38811): tablet 2ea8c9f679ed4192b0a498e6e6740a4e (table test-table [id=4135d7e0d31141e5838d5f7c893b5c06]) successfully deleted
I20260808 03:02:10.735466 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 26238
I20260808 03:02:10.764091 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 26434
Aug 08 03:02:10 dist-test-slave-6ntk krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 03:02:10.782570 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.765014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52412 (local address 127.0.205.130:42459)
0808 03:02:10.765196 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.765200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.765254 (+ 54us) server_negotiation.cc:407] Connection header received
0808 03:02:10.765477 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.765481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.765548 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.765641 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:10.767354 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.768136 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.770247 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.770751 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.770857 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.776269 (+ 5412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:10.776294 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:10.776314 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:10.776354 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:10.779348 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:10.780165 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:10.780174 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:10.780176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:10.780316 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:10.780732 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:10.780736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:10.780738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:10.781134 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:10.781424 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.781860 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.782148 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:02:10.783213 26607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:10.875896 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 26434
I20260808 03:02:10.907047 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 26625
Aug 08 03:02:10 dist-test-slave-6ntk krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 03:02:10.924324 26827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:10.907924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56094 (local address 127.0.205.131:38811)
0808 03:02:10.908183 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:10.908189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:10.908211 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:02:10.908318 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:10.908324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:10.908407 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:10.908524 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:10.909933 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.910892 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.912231 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:10.912703 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:10.912860 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:10.917845 (+ 4985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:10.917862 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:10.917934 (+ 72us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:10.917986 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:10.921173 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:10.921992 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:10.922001 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:10.922003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:10.922175 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:10.922601 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:10.922606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:10.922609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:10.923004 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:10.923366 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:10.923685 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 03:02:10.923936 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260808 03:02:10.925128 26794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:11.050374 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 26625
I20260808 03:02:11.079981 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 26100
I20260808 03:02:11.081285 26198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:11.194854 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 26100
2026-08-08T03:02:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5173 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T03:02:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:11Z Disabled control of system clock
I20260808 03:02:11.297590 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:46873
--webserver_interface=127.0.205.190
--webserver_port=0
--builtin_ntp_servers=127.0.205.148:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:46873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:02:11.665624 26890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:11.666132 26890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:11.666258 26890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:11.673928 26890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:11.674059 26890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:11.674115 26890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:11.674191 26890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:11.674244 26890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:11.683830 26890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:46873
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:46873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:11.685225 26890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:11.687268 26890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:11.700476 26902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:11.700491 26907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:11.700426 26901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:11.701467 26890 server_base.cc:1034] running on GCE node
I20260808 03:02:11.702359 26890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:11.703781 26890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:11.704984 26890 hybrid_clock.cc:648] HybridClock initialized: now 1786158131704932 us; error 43 us; skew 500 ppm
I20260808 03:02:11.705348 26890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:11.707885 26890 webserver.cc:460] Webserver started at http://127.0.205.190:38101/ using document root <none> and password file <none>
I20260808 03:02:11.708599 26890 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:11.708717 26890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:11.709060 26890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:11.712117 26890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "0a55773ed21f467f8f0b2c9c47d7f574"
format_stamp: "Formatted at 2026-08-08 03:02:11 on dist-test-slave-6ntk"
server_key: "4a463f8981b23119268b6872acc0b84f"
server_key_iv: "54cc11faefbfa0ac4f324965670a274b"
server_key_version: "encryptionkey@0"
I20260808 03:02:11.713006 26890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "0a55773ed21f467f8f0b2c9c47d7f574"
format_stamp: "Formatted at 2026-08-08 03:02:11 on dist-test-slave-6ntk"
server_key: "4a463f8981b23119268b6872acc0b84f"
server_key_iv: "54cc11faefbfa0ac4f324965670a274b"
server_key_version: "encryptionkey@0"
I20260808 03:02:11.718377 26890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 03:02:11.721861 26915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:11.723227 26890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:11.723441 26890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "0a55773ed21f467f8f0b2c9c47d7f574"
format_stamp: "Formatted at 2026-08-08 03:02:11 on dist-test-slave-6ntk"
server_key: "4a463f8981b23119268b6872acc0b84f"
server_key_iv: "54cc11faefbfa0ac4f324965670a274b"
server_key_version: "encryptionkey@0"
I20260808 03:02:11.723666 26890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:11.736186 26890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:11.737056 26890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:11.737417 26890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:11.756568 26890 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:46873
I20260808 03:02:11.756593 27001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:46873 every 8 connection(s)
I20260808 03:02:11.758183 26890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:11.762275 27003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:11.768275 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 26890
I20260808 03:02:11.768488 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:11.768967 822 external_mini_cluster.cc:1428] Setting key 606c15a3ab981b330ca1425886ea9265
I20260808 03:02:11.773820 27003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574: Bootstrap starting.
I20260808 03:02:11.777537 27003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:11.778872 27003 log.cc:824] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:11.782312 27003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574: No bootstrap required, opened a new log
I20260808 03:02:11.783268 27008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:11.771056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36856 (local address 127.0.205.190:46873)
0808 03:02:11.772147 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:11.772159 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:11.772202 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:02:11.773948 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:11.773972 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:11.774475 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:11.774778 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:11.775464 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:11.776724 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:11.778732 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:11.779631 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:11.779834 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:11.780917 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:11.780925 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:11.780938 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:11.781468 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:11.781541 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:11.781562 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:11.781659 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:11.781954 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 03:02:11.782289 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":415,"threads_started":1}
I20260808 03:02:11.789971 27003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a55773ed21f467f8f0b2c9c47d7f574" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46873 } }
I20260808 03:02:11.790351 27003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:11.790489 27003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a55773ed21f467f8f0b2c9c47d7f574, State: Initialized, Role: FOLLOWER
I20260808 03:02:11.791072 27003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a55773ed21f467f8f0b2c9c47d7f574" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46873 } }
I20260808 03:02:11.791365 27003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:11.791509 27003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:11.791697 27003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:11.793920 27003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a55773ed21f467f8f0b2c9c47d7f574" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46873 } }
I20260808 03:02:11.794435 27003 leader_election.cc:302] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a55773ed21f467f8f0b2c9c47d7f574; no voters:
I20260808 03:02:11.795029 27003 leader_election.cc:288] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:11.795171 27010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:11.795949 27010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [term 1 LEADER]: Becoming Leader. State: Replica: 0a55773ed21f467f8f0b2c9c47d7f574, State: Running, Role: LEADER
I20260808 03:02:11.796428 27010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a55773ed21f467f8f0b2c9c47d7f574" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46873 } }
I20260808 03:02:11.797355 27003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:11.799587 27012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a55773ed21f467f8f0b2c9c47d7f574. Latest consensus state: current_term: 1 leader_uuid: "0a55773ed21f467f8f0b2c9c47d7f574" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a55773ed21f467f8f0b2c9c47d7f574" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46873 } } }
I20260808 03:02:11.799875 27011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a55773ed21f467f8f0b2c9c47d7f574" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a55773ed21f467f8f0b2c9c47d7f574" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 46873 } } }
I20260808 03:02:11.800127 27012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:11.800196 27011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:11.802173 27022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:11.806874 27022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:11.815804 27022 catalog_manager.cc:1347] Generated new cluster ID: c070795bae2145ecaa964aacdf624964
I20260808 03:02:11.815939 27022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:11.827258 27022 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:11.829218 27022 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:11.839694 27022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574: Generated new TSK 0
I20260808 03:02:11.840848 27022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:11.859607 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46873
--builtin_ntp_servers=127.0.205.148:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:02:12.220543 27038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:12.220907 27038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:12.221077 27038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:12.228741 27038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:12.228890 27038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:12.229069 27038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:02:12.238646 27038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:12.240075 27038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:12.242255 27038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:12.258388 27050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:12.258466 27048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:12.258651 27045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:12.259552 27038 server_base.cc:1034] running on GCE node
I20260808 03:02:12.260294 27038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:12.261303 27038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:12.262554 27038 hybrid_clock.cc:648] HybridClock initialized: now 1786158132262501 us; error 70 us; skew 500 ppm
I20260808 03:02:12.262910 27038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:12.265434 27038 webserver.cc:460] Webserver started at http://127.0.205.129:42991/ using document root <none> and password file <none>
I20260808 03:02:12.266299 27038 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:12.266449 27038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:12.266793 27038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:12.269953 27038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "c61d571cb7644c7094d174ec4a619dd5"
format_stamp: "Formatted at 2026-08-08 03:02:12 on dist-test-slave-6ntk"
server_key: "a5234c5015b5fcaf783c861f67dc3efc"
server_key_iv: "73ea0ffc36eb183ca6e2c6c15f58e2f8"
server_key_version: "encryptionkey@0"
I20260808 03:02:12.270844 27038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "c61d571cb7644c7094d174ec4a619dd5"
format_stamp: "Formatted at 2026-08-08 03:02:12 on dist-test-slave-6ntk"
server_key: "a5234c5015b5fcaf783c861f67dc3efc"
server_key_iv: "73ea0ffc36eb183ca6e2c6c15f58e2f8"
server_key_version: "encryptionkey@0"
I20260808 03:02:12.276388 27038 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260808 03:02:12.280227 27058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:12.281661 27038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:02:12.281860 27038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "c61d571cb7644c7094d174ec4a619dd5"
format_stamp: "Formatted at 2026-08-08 03:02:12 on dist-test-slave-6ntk"
server_key: "a5234c5015b5fcaf783c861f67dc3efc"
server_key_iv: "73ea0ffc36eb183ca6e2c6c15f58e2f8"
server_key_version: "encryptionkey@0"
I20260808 03:02:12.282153 27038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:12.300130 27038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:12.301043 27038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:12.301370 27038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:12.302634 27038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:12.304145 27038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:12.304288 27038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:12.304418 27038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:12.304509 27038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:12.341382 27038 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:44091
I20260808 03:02:12.341481 27234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:44091 every 8 connection(s)
I20260808 03:02:12.342819 27038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:02:12.344138 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 27038
I20260808 03:02:12.344322 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 03:02:12.344916 822 external_mini_cluster.cc:1428] Setting key 8f09667a3f9fd6855216ac354df614d6
I20260808 03:02:12.348384 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46873
--builtin_ntp_servers=127.0.205.148:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:02:12.360080 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:12.346011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:60497 (local address 127.0.205.190:46873)
0808 03:02:12.348779 (+ 2768us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:12.348785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:12.348808 (+ 23us) server_negotiation.cc:407] Connection header received
0808 03:02:12.351461 (+ 2653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:12.351467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:12.351571 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:12.351697 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:12.353339 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:12.354181 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:12.356758 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:12.357196 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:12.357320 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:12.359242 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:12.359249 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:12.359253 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:12.359275 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:12.359321 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:12.359336 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:12.359417 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:12.359719 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 03:02:12.359810 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2624,"thread_start_us":2497,"threads_started":1}
I20260808 03:02:12.361650 27242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:12.346812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:46873 (local address 127.0.205.129:60497)
0808 03:02:12.348088 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:12.348154 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:12.351065 (+ 2911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:12.351994 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:12.352010 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:12.352144 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:12.353023 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:12.353052 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:12.354342 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:12.354351 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:12.356576 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:12.356595 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:12.357462 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:12.357471 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:12.357823 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:12.358822 (+ 999us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:02:12.358828 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:12.358887 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:02:12.359006 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:12.359445 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:12.359461 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:12.359472 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 03:02:12.359674 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 03:02:12.360154 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":392,"threads_started":1}
I20260808 03:02:12.363504 27236 heartbeater.cc:347] Connected to a master server at 127.0.205.190:46873
I20260808 03:02:12.363855 27236 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:12.364661 27236 heartbeater.cc:511] Master 127.0.205.190:46873 requested a full tablet report, sending...
I20260808 03:02:12.366993 26938 ts_manager.cc:194] Registered new tserver with Master: c61d571cb7644c7094d174ec4a619dd5 (127.0.205.129:44091)
I20260808 03:02:12.368551 26938 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.129:60497
W20260808 03:02:12.715250 27244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:12.715613 27244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:12.715787 27244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:12.723573 27244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:12.723742 27244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:12.723932 27244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:02:12.733707 27244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:12.735174 27244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:12.737087 27244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:12.751240 27256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:12.751296 27260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:12.751271 27255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:12.752086 27244 server_base.cc:1034] running on GCE node
I20260808 03:02:12.752812 27244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:12.753945 27244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:12.755195 27244 hybrid_clock.cc:648] HybridClock initialized: now 1786158132755146 us; error 48 us; skew 500 ppm
I20260808 03:02:12.755546 27244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:12.758160 27244 webserver.cc:460] Webserver started at http://127.0.205.130:33633/ using document root <none> and password file <none>
I20260808 03:02:12.758888 27244 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:12.759025 27244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:12.759451 27244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:12.762451 27244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2"
format_stamp: "Formatted at 2026-08-08 03:02:12 on dist-test-slave-6ntk"
server_key: "8c52e5ac9c9b0c549f1dd9a84bb0d80d"
server_key_iv: "45795f24c04049a9e51b311c8475f060"
server_key_version: "encryptionkey@0"
I20260808 03:02:12.763378 27244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2"
format_stamp: "Formatted at 2026-08-08 03:02:12 on dist-test-slave-6ntk"
server_key: "8c52e5ac9c9b0c549f1dd9a84bb0d80d"
server_key_iv: "45795f24c04049a9e51b311c8475f060"
server_key_version: "encryptionkey@0"
I20260808 03:02:12.768988 27244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 03:02:12.772620 27270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:12.774092 27244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:02:12.774231 27244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2"
format_stamp: "Formatted at 2026-08-08 03:02:12 on dist-test-slave-6ntk"
server_key: "8c52e5ac9c9b0c549f1dd9a84bb0d80d"
server_key_iv: "45795f24c04049a9e51b311c8475f060"
server_key_version: "encryptionkey@0"
I20260808 03:02:12.774436 27244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:12.793206 27244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:12.794065 27244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:12.794333 27244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:12.795284 27244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:12.796729 27244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:12.796818 27244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:12.796900 27244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:12.796955 27244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:12.831305 27244 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:34811
I20260808 03:02:12.831478 27440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:34811 every 8 connection(s)
I20260808 03:02:12.832726 27244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:02:12.834276 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 27244
I20260808 03:02:12.834496 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 03:02:12.834986 822 external_mini_cluster.cc:1428] Setting key a678cf86b6b1267eb537f382619af227
I20260808 03:02:12.838446 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46873
--builtin_ntp_servers=127.0.205.148:35177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:02:12.849009 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:12.835789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:53433 (local address 127.0.205.190:46873)
0808 03:02:12.835983 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:12.835990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:12.837665 (+ 1675us) server_negotiation.cc:407] Connection header received
0808 03:02:12.840619 (+ 2954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:12.840624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:12.840693 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:12.840784 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:12.842286 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:12.843024 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:12.845562 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:12.846083 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:12.846183 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:12.848027 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:12.848036 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:12.848043 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:12.848069 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:12.848114 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:12.848131 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:12.848284 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:12.848498 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 03:02:12.848616 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:02:12.850435 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:12.836340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:46873 (local address 127.0.205.130:53433)
0808 03:02:12.837492 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:12.837548 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:12.840286 (+ 2738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:12.841068 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:12.841085 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:12.841215 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:12.842093 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:12.842122 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:12.843189 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:12.843197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:12.845400 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:12.845419 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:12.846320 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:12.846331 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:12.846687 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:12.847611 (+ 924us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:02:12.847616 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:12.847673 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:02:12.847793 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:12.848242 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:12.848255 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:12.848266 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 03:02:12.848471 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 03:02:12.848932 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":333,"threads_started":1}
I20260808 03:02:12.851974 27443 heartbeater.cc:347] Connected to a master server at 127.0.205.190:46873
I20260808 03:02:12.852306 27443 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:12.853156 27443 heartbeater.cc:511] Master 127.0.205.190:46873 requested a full tablet report, sending...
I20260808 03:02:12.854875 26938 ts_manager.cc:194] Registered new tserver with Master: 1d1a740fe1e64fe78c57c3f5205f4eb2 (127.0.205.130:34811)
I20260808 03:02:12.855722 26938 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.130:53433
W20260808 03:02:13.209537 27447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:13.209995 27447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:13.210163 27447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:13.220907 27447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:13.221077 27447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:13.221262 27447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:02:13.232803 27447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:35177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:46873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:13.234270 27447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:13.236133 27447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:13.250051 27453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:13.249972 27458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:13.249972 27456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:13.250914 27447 server_base.cc:1034] running on GCE node
I20260808 03:02:13.251570 27447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:13.252698 27447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:13.253968 27447 hybrid_clock.cc:648] HybridClock initialized: now 1786158133253871 us; error 83 us; skew 500 ppm
I20260808 03:02:13.254402 27447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:13.262396 27447 webserver.cc:460] Webserver started at http://127.0.205.131:34731/ using document root <none> and password file <none>
I20260808 03:02:13.263223 27447 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:13.263348 27447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:13.263643 27447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:13.266793 27447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "7847d33dfebe44328aa1a867bc276204"
format_stamp: "Formatted at 2026-08-08 03:02:13 on dist-test-slave-6ntk"
server_key: "1c06179a74070d8c4fc160d135373fe6"
server_key_iv: "7f1b30d62e71083a032ccbc680436580"
server_key_version: "encryptionkey@0"
I20260808 03:02:13.267617 27447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "7847d33dfebe44328aa1a867bc276204"
format_stamp: "Formatted at 2026-08-08 03:02:13 on dist-test-slave-6ntk"
server_key: "1c06179a74070d8c4fc160d135373fe6"
server_key_iv: "7f1b30d62e71083a032ccbc680436580"
server_key_version: "encryptionkey@0"
I20260808 03:02:13.273099 27447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:02:13.276966 27469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:13.278488 27447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:02:13.278640 27447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "7847d33dfebe44328aa1a867bc276204"
format_stamp: "Formatted at 2026-08-08 03:02:13 on dist-test-slave-6ntk"
server_key: "1c06179a74070d8c4fc160d135373fe6"
server_key_iv: "7f1b30d62e71083a032ccbc680436580"
server_key_version: "encryptionkey@0"
I20260808 03:02:13.278836 27447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:13.305815 27447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:13.306712 27447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:13.306985 27447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:13.307977 27447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:13.309466 27447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:13.309556 27447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:13.309643 27447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:13.309702 27447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:13.345283 27447 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:46121
I20260808 03:02:13.345408 27639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:46121 every 8 connection(s)
I20260808 03:02:13.346972 27447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:02:13.355175 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 27447
I20260808 03:02:13.355389 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 03:02:13.355834 822 external_mini_cluster.cc:1428] Setting key 362c3db05e2d27a665eb4afb1f1d15cc
I20260808 03:02:13.361511 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.350070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:55569 (local address 127.0.205.190:46873)
0808 03:02:13.350661 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.350667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.351695 (+ 1028us) server_negotiation.cc:407] Connection header received
0808 03:02:13.353720 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.353727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.353831 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.354000 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:13.355261 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.356380 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.358601 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.359209 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.359333 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.360729 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:13.360734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:13.360738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:13.360757 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:13.360800 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:13.360812 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:13.360888 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.361094 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.361185 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":355,"threads_started":1}
I20260808 03:02:13.362571 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.350547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:46873 (local address 127.0.205.131:55569)
0808 03:02:13.351512 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.351553 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:13.353414 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:13.354308 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:13.354318 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:13.354414 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:13.355064 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.355083 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.356605 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.356612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.358447 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.358460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.359422 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.359428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.359630 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.360379 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:02:13.360382 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:13.360424 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:02:13.360505 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:13.360902 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:13.360911 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:13.360916 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 03:02:13.361062 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 03:02:13.361360 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":298,"threads_started":1}
I20260808 03:02:13.364120 27641 heartbeater.cc:347] Connected to a master server at 127.0.205.190:46873
I20260808 03:02:13.364424 27641 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:13.365167 27641 heartbeater.cc:511] Master 127.0.205.190:46873 requested a full tablet report, sending...
I20260808 03:02:13.366711 26938 ts_manager.cc:194] Registered new tserver with Master: 7847d33dfebe44328aa1a867bc276204 (127.0.205.131:46121)
I20260808 03:02:13.367550 26938 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.131:55569
I20260808 03:02:13.371678 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:02:13.371789 27236 heartbeater.cc:503] Master 127.0.205.190:46873 was elected leader, sending a full tablet report...
I20260808 03:02:13.382380 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.376025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.0.205.190:46873)
0808 03:02:13.376208 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.376213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.376732 (+ 519us) server_negotiation.cc:407] Connection header received
0808 03:02:13.376868 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.376871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.376930 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.377013 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:13.377541 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.378504 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.379907 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.380288 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.380392 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.381501 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:13.381507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:13.381510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:13.381531 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:13.381563 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:13.381576 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:13.381660 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.381807 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.381969 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:02:13.386643 26938 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:02:13.393577 26938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:02:13.419035 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.406448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:46121 (local address 127.0.0.1:37434)
0808 03:02:13.407989 (+ 1541us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.408029 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:13.408201 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:13.410447 (+ 2246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:13.410458 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:13.410501 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:13.411074 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.411100 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.413196 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.413201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.415666 (+ 2465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.415687 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.417990 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.417998 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.418207 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.418263 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 03:02:13.418413 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 03:02:13.418620 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1223,"mutex_wait_us":87,"thread_start_us":2458,"threads_started":1}
I20260808 03:02:13.419697 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.407026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44091 (local address 127.0.0.1:48958)
0808 03:02:13.409196 (+ 2170us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.409218 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:13.409336 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:13.412758 (+ 3422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:13.412763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:13.412789 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:13.413293 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.413308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.415162 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.415168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.416822 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.416835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.418805 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.418811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.419003 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.419046 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 03:02:13.419141 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 03:02:13.419276 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2034,"mutex_wait_us":4,"thread_start_us":833,"threads_started":1}
I20260808 03:02:13.419734 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.406791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.0.205.131:46121)
0808 03:02:13.408942 (+ 2151us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.408965 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.409008 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:02:13.409132 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.409144 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.409752 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.410013 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:13.411381 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.413033 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.415908 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.417350 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.417509 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.417836 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.419160 (+ 1324us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.419348 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1926,"thread_start_us":1717,"threads_started":1}
I20260808 03:02:13.420091 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.407675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.0.205.129:44091)
0808 03:02:13.412181 (+ 4506us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.412189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.412219 (+ 30us) server_negotiation.cc:407] Connection header received
0808 03:02:13.412321 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.412332 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.412541 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.412772 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:13.413532 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.414992 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.417030 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.418538 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.418737 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.419037 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.419607 (+ 570us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.419796 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4334,"thread_start_us":4212,"threads_started":1}
I20260808 03:02:13.420907 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.409851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:34811 (local address 127.0.0.1:46918)
0808 03:02:13.411086 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.411110 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:13.411258 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:13.412399 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:13.412403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:13.412429 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:13.412776 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.412787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.414756 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.414762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.417053 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.417068 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.419452 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.419464 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.419646 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.419685 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 03:02:13.420462 (+ 777us) client_negotiation.cc:240] Negotiation successful
0808 03:02:13.420597 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"mutex_wait_us":72,"thread_start_us":886,"threads_started":1}
I20260808 03:02:13.421567 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.409591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46918 (local address 127.0.205.130:34811)
0808 03:02:13.411769 (+ 2178us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.411780 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.411815 (+ 35us) server_negotiation.cc:407] Connection header received
0808 03:02:13.411926 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.411934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.412179 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.412404 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:13.413003 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.414604 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.417253 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.418692 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.418822 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.419014 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.421185 (+ 2171us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.421330 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2034,"thread_start_us":1943,"threads_started":1}
I20260808 03:02:13.424036 27539 tablet_service.cc:1467] Processing CreateTablet for tablet c4afaf39764e49f78bc659519a5765ce (DEFAULT_TABLE table=test-table [id=d44e03055f3d44e19f90905a0635a463]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:13.424037 27136 tablet_service.cc:1467] Processing CreateTablet for tablet c4afaf39764e49f78bc659519a5765ce (DEFAULT_TABLE table=test-table [id=d44e03055f3d44e19f90905a0635a463]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:13.424782 27338 tablet_service.cc:1467] Processing CreateTablet for tablet c4afaf39764e49f78bc659519a5765ce (DEFAULT_TABLE table=test-table [id=d44e03055f3d44e19f90905a0635a463]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:13.425621 27136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4afaf39764e49f78bc659519a5765ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:13.425621 27539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4afaf39764e49f78bc659519a5765ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:13.426038 27338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4afaf39764e49f78bc659519a5765ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:13.438041 27676 tablet_bootstrap.cc:492] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: Bootstrap starting.
I20260808 03:02:13.440745 27677 tablet_bootstrap.cc:492] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5: Bootstrap starting.
I20260808 03:02:13.442010 27676 tablet_bootstrap.cc:654] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:13.442593 27679 tablet_bootstrap.cc:492] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: Bootstrap starting.
I20260808 03:02:13.443589 27676 log.cc:824] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:13.444736 27677 tablet_bootstrap.cc:654] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:13.446077 27677 log.cc:824] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:13.447193 27676 tablet_bootstrap.cc:492] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: No bootstrap required, opened a new log
I20260808 03:02:13.447530 27676 ts_tablet_manager.cc:1397] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260808 03:02:13.448249 27679 tablet_bootstrap.cc:654] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:13.448956 27677 tablet_bootstrap.cc:492] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5: No bootstrap required, opened a new log
I20260808 03:02:13.449226 27677 ts_tablet_manager.cc:1397] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 03:02:13.449512 27679 log.cc:824] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:13.452728 27679 tablet_bootstrap.cc:492] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: No bootstrap required, opened a new log
I20260808 03:02:13.453019 27679 ts_tablet_manager.cc:1397] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.004s
I20260808 03:02:13.457366 27677 raft_consensus.cc:348] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 } } peers { permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 } } peers { permanent_uuid: "c61d571cb7644c7094d174ec4a619dd5" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44091 } }
I20260808 03:02:13.457985 27677 raft_consensus.cc:374] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:13.458132 27677 raft_consensus.cc:729] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c61d571cb7644c7094d174ec4a619dd5, State: Initialized, Role: FOLLOWER
I20260808 03:02:13.458137 27676 raft_consensus.cc:348] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 } } peers { permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 } } peers { permanent_uuid: "c61d571cb7644c7094d174ec4a619dd5" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44091 } }
I20260808 03:02:13.458599 27676 raft_consensus.cc:374] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:13.458714 27676 raft_consensus.cc:729] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1a740fe1e64fe78c57c3f5205f4eb2, State: Initialized, Role: FOLLOWER
I20260808 03:02:13.458889 27677 consensus_queue.cc:260] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 } } peers { permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 } } peers { permanent_uuid: "c61d571cb7644c7094d174ec4a619dd5" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44091 } }
I20260808 03:02:13.459317 27676 consensus_queue.cc:260] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 } } peers { permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 } } peers { permanent_uuid: "c61d571cb7644c7094d174ec4a619dd5" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44091 } }
I20260808 03:02:13.460953 27677 ts_tablet_manager.cc:1428] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 03:02:13.462041 27443 heartbeater.cc:503] Master 127.0.205.190:46873 was elected leader, sending a full tablet report...
I20260808 03:02:13.462669 27676 ts_tablet_manager.cc:1428] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260808 03:02:13.464447 27679 raft_consensus.cc:348] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 } } peers { permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 } } peers { permanent_uuid: "c61d571cb7644c7094d174ec4a619dd5" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44091 } }
I20260808 03:02:13.464872 27679 raft_consensus.cc:374] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:13.465004 27679 raft_consensus.cc:729] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7847d33dfebe44328aa1a867bc276204, State: Initialized, Role: FOLLOWER
I20260808 03:02:13.465631 27679 consensus_queue.cc:260] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 } } peers { permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 } } peers { permanent_uuid: "c61d571cb7644c7094d174ec4a619dd5" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44091 } }
I20260808 03:02:13.470212 27641 heartbeater.cc:503] Master 127.0.205.190:46873 was elected leader, sending a full tablet report...
I20260808 03:02:13.470824 27679 ts_tablet_manager.cc:1428] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: Time spent starting tablet: real 0.018s user 0.016s sys 0.003s
I20260808 03:02:13.518256 27688 raft_consensus.cc:482] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:02:13.518607 27688 raft_consensus.cc:504] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 } } peers { permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 } } peers { permanent_uuid: "c61d571cb7644c7094d174ec4a619dd5" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44091 } }
I20260808 03:02:13.520355 27688 leader_election.cc:288] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7847d33dfebe44328aa1a867bc276204 (127.0.205.131:46121), 1d1a740fe1e64fe78c57c3f5205f4eb2 (127.0.205.130:34811)
I20260808 03:02:13.527580 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.520868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:35127 (local address 127.0.205.131:46121)
0808 03:02:13.521103 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.521111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.521924 (+ 813us) server_negotiation.cc:407] Connection header received
0808 03:02:13.522112 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.522118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.522212 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.522341 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:13.523020 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.524008 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.525804 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.526652 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.526837 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.526909 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.527187 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.527314 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:02:13.527616 27692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.520889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:46121 (local address 127.0.205.129:35127)
0808 03:02:13.521645 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.521666 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:13.521900 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:13.522453 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:13.522457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:13.522482 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:13.522847 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.522856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.524150 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.524156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.525607 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.525619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.526840 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.526846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.527014 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.527046 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 03:02:13.527159 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 03:02:13.527288 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":453,"threads_started":1}
I20260808 03:02:13.528757 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.522182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:42791 (local address 127.0.205.130:34811)
0808 03:02:13.522370 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.522375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.523274 (+ 899us) server_negotiation.cc:407] Connection header received
0808 03:02:13.523418 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.523423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.523510 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.523605 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:13.524213 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.524912 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.526750 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.527633 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.527857 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.527934 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.528367 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.528511 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:02:13.528776 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.522112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:34811 (local address 127.0.205.129:42791)
0808 03:02:13.523152 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.523169 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:13.523290 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:13.523669 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:13.523673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:13.523696 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:13.524000 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.524009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.525073 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.525080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.526531 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:13.526544 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.528056 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.528062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:13.528186 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.528208 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 03:02:13.528317 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 03:02:13.528448 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":655,"threads_started":1}
I20260808 03:02:13.528633 27569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4afaf39764e49f78bc659519a5765ce" candidate_uuid: "c61d571cb7644c7094d174ec4a619dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7847d33dfebe44328aa1a867bc276204" is_pre_election: true
I20260808 03:02:13.529238 27569 raft_consensus.cc:2454] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c61d571cb7644c7094d174ec4a619dd5 in term 0.
I20260808 03:02:13.529918 27365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4afaf39764e49f78bc659519a5765ce" candidate_uuid: "c61d571cb7644c7094d174ec4a619dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" is_pre_election: true
I20260808 03:02:13.530036 27060 leader_election.cc:302] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7847d33dfebe44328aa1a867bc276204, c61d571cb7644c7094d174ec4a619dd5; no voters:
I20260808 03:02:13.530532 27688 raft_consensus.cc:2781] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:02:13.530519 27365 raft_consensus.cc:2454] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c61d571cb7644c7094d174ec4a619dd5 in term 0.
I20260808 03:02:13.530750 27688 raft_consensus.cc:482] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:02:13.530881 27688 raft_consensus.cc:3037] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:13.533421 27688 raft_consensus.cc:504] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 } } peers { permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 } } peers { permanent_uuid: "c61d571cb7644c7094d174ec4a619dd5" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44091 } }
I20260808 03:02:13.534104 27688 leader_election.cc:288] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [CANDIDATE]: Term 1 election: Requested vote from peers 7847d33dfebe44328aa1a867bc276204 (127.0.205.131:46121), 1d1a740fe1e64fe78c57c3f5205f4eb2 (127.0.205.130:34811)
I20260808 03:02:13.534622 27569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4afaf39764e49f78bc659519a5765ce" candidate_uuid: "c61d571cb7644c7094d174ec4a619dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7847d33dfebe44328aa1a867bc276204"
I20260808 03:02:13.534890 27569 raft_consensus.cc:3037] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:13.535035 27365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4afaf39764e49f78bc659519a5765ce" candidate_uuid: "c61d571cb7644c7094d174ec4a619dd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2"
I20260808 03:02:13.535352 27365 raft_consensus.cc:3037] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:13.537281 27569 raft_consensus.cc:2454] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c61d571cb7644c7094d174ec4a619dd5 in term 1.
I20260808 03:02:13.537763 27060 leader_election.cc:302] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7847d33dfebe44328aa1a867bc276204, c61d571cb7644c7094d174ec4a619dd5; no voters:
I20260808 03:02:13.537787 27365 raft_consensus.cc:2454] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c61d571cb7644c7094d174ec4a619dd5 in term 1.
I20260808 03:02:13.538117 27688 raft_consensus.cc:2781] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:13.538648 27688 raft_consensus.cc:686] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 1 LEADER]: Becoming Leader. State: Replica: c61d571cb7644c7094d174ec4a619dd5, State: Running, Role: LEADER
I20260808 03:02:13.539137 27688 consensus_queue.cc:237] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 } } peers { permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 } } peers { permanent_uuid: "c61d571cb7644c7094d174ec4a619dd5" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44091 } }
I20260808 03:02:13.543813 26938 catalog_manager.cc:5495] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c61d571cb7644c7094d174ec4a619dd5 (127.0.205.129). New cstate: current_term: 1 leader_uuid: "c61d571cb7644c7094d174ec4a619dd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c61d571cb7644c7094d174ec4a619dd5" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 44091 } health_report { overall_health: HEALTHY } } }
I20260808 03:02:13.574515 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.568826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.0.205.129:44091)
0808 03:02:13.569038 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.569042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.569084 (+ 42us) server_negotiation.cc:407] Connection header received
0808 03:02:13.569343 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.569347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.569448 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.569598 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:02:13.569985 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.570702 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.572182 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.572601 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.572695 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.573274 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:02:13.573914 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:02:13.574060 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.574185 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.574273 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 03:02:13.583247 27569 raft_consensus.cc:1261] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204 [term 1 FOLLOWER]: Refusing update from remote peer c61d571cb7644c7094d174ec4a619dd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:13.583247 27365 raft_consensus.cc:1261] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2 [term 1 FOLLOWER]: Refusing update from remote peer c61d571cb7644c7094d174ec4a619dd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:13.584268 27694 consensus_queue.cc:1035] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1a740fe1e64fe78c57c3f5205f4eb2" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 34811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:13.584695 27688 consensus_queue.cc:1035] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7847d33dfebe44328aa1a867bc276204" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 03:02:13.584949 27444 tablet.cc:2369] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:02:13.595393 27239 tablet.cc:2369] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:02:13.599917 27697 mvcc.cc:205] Tried to move back new op lower bound from 7316103715146231808 to 7316103714980069376. Current Snapshot: MvccSnapshot[applied={T|T < 7316103715146231808 or (T in {7316103715146231808})}]
I20260808 03:02:13.610967 26937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:13.611256 26937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:13.615401 26937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a55773ed21f467f8f0b2c9c47d7f574: Sending DeleteTablet for 3 replicas of tablet c4afaf39764e49f78bc659519a5765ce
I20260808 03:02:13.616827 27539 tablet_service.cc:1514] Processing DeleteTablet for tablet c4afaf39764e49f78bc659519a5765ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:13 UTC) from {username='slave'} at 127.0.0.1:37434
I20260808 03:02:13.616804 27338 tablet_service.cc:1514] Processing DeleteTablet for tablet c4afaf39764e49f78bc659519a5765ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:13 UTC) from {username='slave'} at 127.0.0.1:46918
I20260808 03:02:13.616837 27136 tablet_service.cc:1514] Processing DeleteTablet for tablet c4afaf39764e49f78bc659519a5765ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:13 UTC) from {username='slave'} at 127.0.0.1:48958
I20260808 03:02:13.618716 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 27038
I20260808 03:02:13.628326 27710 tablet_replica.cc:330] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: stopping tablet replica
I20260808 03:02:13.629586 27710 raft_consensus.cc:2229] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:13.631022 27710 raft_consensus.cc:2258] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:13.634099 27710 ts_tablet_manager.cc:1905] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:13.636684 27714 tablet_replica.cc:330] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: stopping tablet replica
I20260808 03:02:13.636958 27711 tablet_replica.cc:330] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5: stopping tablet replica
I20260808 03:02:13.637203 27714 raft_consensus.cc:2229] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:13.637379 27711 raft_consensus.cc:2229] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:02:13.637800 27714 raft_consensus.cc:2258] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:13.638134 27711 raft_consensus.cc:2258] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:13.640312 27711 ts_tablet_manager.cc:1905] T c4afaf39764e49f78bc659519a5765ce P c61d571cb7644c7094d174ec4a619dd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:13.640645 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.621944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48970 (local address 127.0.205.129:44091)
0808 03:02:13.622651 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.622661 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.625318 (+ 2657us) server_negotiation.cc:407] Connection header received
0808 03:02:13.625529 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.625539 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.625693 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.625827 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:13.629020 (+ 3193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.630726 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.633680 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.634746 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.634907 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.638705 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:13.638717 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:13.638734 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:13.639602 (+ 868us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:13.639680 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:13.639707 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:13.639808 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.640089 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.640227 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519}
I20260808 03:02:13.641925 27229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:13.643431 27714 ts_tablet_manager.cc:1905] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:13.643796 27710 ts_tablet_manager.cc:1918] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:13.644011 27710 log.cc:1196] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/c4afaf39764e49f78bc659519a5765ce
I20260808 03:02:13.644714 27710 ts_tablet_manager.cc:1939] T c4afaf39764e49f78bc659519a5765ce P 1d1a740fe1e64fe78c57c3f5205f4eb2: Deleting consensus metadata
I20260808 03:02:13.647125 26918 catalog_manager.cc:4841] TS 1d1a740fe1e64fe78c57c3f5205f4eb2 (127.0.205.130:34811): tablet c4afaf39764e49f78bc659519a5765ce (table test-table [id=d44e03055f3d44e19f90905a0635a463]) successfully deleted
I20260808 03:02:13.650046 27714 ts_tablet_manager.cc:1918] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:13.650326 27714 log.cc:1196] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/c4afaf39764e49f78bc659519a5765ce
I20260808 03:02:13.650864 27714 ts_tablet_manager.cc:1939] T c4afaf39764e49f78bc659519a5765ce P 7847d33dfebe44328aa1a867bc276204: Deleting consensus metadata
I20260808 03:02:13.652511 26918 catalog_manager.cc:4841] TS 7847d33dfebe44328aa1a867bc276204 (127.0.205.131:46121): tablet c4afaf39764e49f78bc659519a5765ce (table test-table [id=d44e03055f3d44e19f90905a0635a463]) successfully deleted
I20260808 03:02:13.747387 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 27038
W20260808 03:02:13.775442 26917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.205.129:44091 (error 108)
I20260808 03:02:13.775738 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 27244
I20260808 03:02:13.777674 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.777134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44091 (local address 127.0.0.1:48972)
0808 03:02:13.777333 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.777491 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 03:02:13.777999 26917 catalog_manager.cc:4584] TS c61d571cb7644c7094d174ec4a619dd5 (127.0.205.129:44091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c4afaf39764e49f78bc659519a5765ce: Network error: Client connection negotiation failed: client connection to 127.0.205.129:44091: connect: Connection refused (error 111)
I20260808 03:02:13.783452 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.776668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46928 (local address 127.0.205.130:34811)
0808 03:02:13.776851 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.776855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.776908 (+ 53us) server_negotiation.cc:407] Connection header received
0808 03:02:13.777081 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.777087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.777210 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.777339 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:13.777955 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.778884 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.780230 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.780674 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.780794 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.781971 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:13.781982 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:13.781997 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:13.782741 (+ 744us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:13.782796 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:13.782819 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:13.782985 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.783061 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.783157 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:02:13.784227 27436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:13.831951 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.831372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44091 (local address 127.0.0.1:48988)
0808 03:02:13.831622 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.831740 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 03:02:13.833096 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.832686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44091 (local address 127.0.0.1:48994)
0808 03:02:13.832850 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.832941 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 03:02:13.877359 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 27244
I20260808 03:02:13.893759 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.893219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44091 (local address 127.0.0.1:49006)
0808 03:02:13.893459 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.893576 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 03:02:13.894855 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.894504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44091 (local address 127.0.0.1:49014)
0808 03:02:13.894666 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.894739 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 03:02:13.900264 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 27447
I20260808 03:02:13.907639 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.901036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.0.205.131:46121)
0808 03:02:13.901229 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:13.901237 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:13.901260 (+ 23us) server_negotiation.cc:407] Connection header received
0808 03:02:13.901487 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:13.901492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:13.901670 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:13.901789 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:13.902243 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.902999 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.904322 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:13.904744 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:13.904890 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:13.906212 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:13.906223 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:13.906239 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:13.906840 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:13.906897 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:13.906918 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:13.907053 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:13.907192 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 03:02:13.907312 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:02:13.908417 27635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:13.975613 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.975045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44091 (local address 127.0.0.1:49018)
0808 03:02:13.975351 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.975452 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260808 03:02:13.976773 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:13.976345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:44091 (local address 127.0.0.1:49022)
0808 03:02:13.976578 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:13.976658 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260808 03:02:13.998258 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 27447
I20260808 03:02:14.022794 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 26890
I20260808 03:02:14.024312 26993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:14.139379 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 26890
2026-08-08T03:02:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2948 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:14 dist-test-slave-6ntk krb5kdc[27728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:14 dist-test-slave-6ntk krb5kdc[27728](info): set up 2 sockets
Aug 08 03:02:14 dist-test-slave-6ntk krb5kdc[27728](info): commencing operation
krb5kdc: starting...
W20260808 03:02:15.495649 822 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:15 dist-test-slave-6ntk krb5kdc[27728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:02:15.825805 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43441
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:43441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:16.185475 27755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:16.185880 27755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:16.186116 27755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:16.195717 27755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:16.195876 27755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:16.195981 27755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:16.196074 27755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:16.196164 27755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:16.207072 27755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:43441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:16.208554 27755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:16.209976 27755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:16.222510 27767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:16.222474 27765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:16.222579 27764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:16.223600 27755 server_base.cc:1034] running on GCE node
I20260808 03:02:16.224483 27755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:16.226114 27755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:16.227370 27755 hybrid_clock.cc:648] HybridClock initialized: now 1786158136227314 us; error 61 us; skew 500 ppm
I20260808 03:02:16.227756 27755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:16 dist-test-slave-6ntk krb5kdc[27728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158136, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:16.231145 27755 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:02:16.232772 27755 webserver.cc:460] Webserver started at http://127.0.205.190:33415/ using document root <none> and password file <none>
I20260808 03:02:16.233525 27755 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:16.233677 27755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:16.234079 27755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:16.237175 27755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "87710e25bb724cb688f236f268821435"
format_stamp: "Formatted at 2026-08-08 03:02:16 on dist-test-slave-6ntk"
server_key: "037085027c573125377cd9b914ef2c65"
server_key_iv: "52995abf935c06e1b9ef82f123221995"
server_key_version: "encryptionkey@0"
I20260808 03:02:16.238092 27755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "87710e25bb724cb688f236f268821435"
format_stamp: "Formatted at 2026-08-08 03:02:16 on dist-test-slave-6ntk"
server_key: "037085027c573125377cd9b914ef2c65"
server_key_iv: "52995abf935c06e1b9ef82f123221995"
server_key_version: "encryptionkey@0"
I20260808 03:02:16.243525 27755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 03:02:16.247231 27776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:16.248720 27755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:16.248915 27755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "87710e25bb724cb688f236f268821435"
format_stamp: "Formatted at 2026-08-08 03:02:16 on dist-test-slave-6ntk"
server_key: "037085027c573125377cd9b914ef2c65"
server_key_iv: "52995abf935c06e1b9ef82f123221995"
server_key_version: "encryptionkey@0"
I20260808 03:02:16.249137 27755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:16.279246 27755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:16.284325 27755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:16.284703 27755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:16.306496 27755 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:43441
I20260808 03:02:16.306500 27850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:43441 every 8 connection(s)
I20260808 03:02:16.308127 27755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:16.308975 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 27755
I20260808 03:02:16.309218 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:16.309826 822 external_mini_cluster.cc:1428] Setting key 295aaf28567d1b0f1d56f3933ec5064f
I20260808 03:02:16.312870 27851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 03:02:16 dist-test-slave-6ntk krb5kdc[27728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:16.327358 27851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435: Bootstrap starting.
I20260808 03:02:16.331952 27851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:16.333293 27851 log.cc:824] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:16.336606 27851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435: No bootstrap required, opened a new log
I20260808 03:02:16.337476 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:16.312088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.0.205.190:43441)
0808 03:02:16.313175 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:16.313187 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:16.313241 (+ 54us) server_negotiation.cc:407] Connection header received
0808 03:02:16.315311 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:16.315337 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:16.315816 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:16.316212 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:16.318479 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:16.319880 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:16.321856 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:16.322671 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:16.322815 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:16.328181 (+ 5366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:16.328194 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:16.328216 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:16.328280 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:16.332653 (+ 4373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:16.333634 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:16.333642 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:16.333650 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:16.333862 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:16.334373 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:16.334378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:16.334382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:16.334993 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:16.335328 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:16.335840 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 03:02:16.336298 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":848,"thread_start_us":484,"threads_started":1}
I20260808 03:02:16.344722 27851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87710e25bb724cb688f236f268821435" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43441 } }
I20260808 03:02:16.345186 27851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:16.345330 27851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87710e25bb724cb688f236f268821435, State: Initialized, Role: FOLLOWER
I20260808 03:02:16.346035 27851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87710e25bb724cb688f236f268821435" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43441 } }
I20260808 03:02:16.346318 27851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:16.346486 27851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:16.346664 27851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:16.349107 27851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87710e25bb724cb688f236f268821435" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43441 } }
I20260808 03:02:16.349686 27851 leader_election.cc:302] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87710e25bb724cb688f236f268821435; no voters:
I20260808 03:02:16.350344 27851 leader_election.cc:288] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:16.350492 27863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:16.351897 27863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [term 1 LEADER]: Becoming Leader. State: Replica: 87710e25bb724cb688f236f268821435, State: Running, Role: LEADER
I20260808 03:02:16.352733 27851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:16.352627 27863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87710e25bb724cb688f236f268821435" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43441 } }
I20260808 03:02:16.355997 27866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87710e25bb724cb688f236f268821435. Latest consensus state: current_term: 1 leader_uuid: "87710e25bb724cb688f236f268821435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87710e25bb724cb688f236f268821435" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43441 } } }
I20260808 03:02:16.356123 27865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87710e25bb724cb688f236f268821435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87710e25bb724cb688f236f268821435" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43441 } } }
I20260808 03:02:16.356483 27866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:16.356612 27865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:16.360070 27874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:16.365965 27874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:16.376225 27874 catalog_manager.cc:1347] Generated new cluster ID: 8cd20630ec394724ae959d5c10f22bfb
I20260808 03:02:16.376394 27874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:16.385787 27874 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:16.388019 27874 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:16.395639 27874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435: Generated new TSK 0
I20260808 03:02:16.396687 27874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 03:02:16.479099 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43441
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:16.846594 27898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:16.846891 27898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:16.846988 27898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:16.854753 27898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:16.854864 27898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:16.854979 27898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:02:16.865197 27898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:16.866576 27898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:16.867875 27898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:16.881762 27909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:16.881810 27910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:16.881733 27912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:16.882733 27898 server_base.cc:1034] running on GCE node
I20260808 03:02:16.883447 27898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:16.884495 27898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:16.885731 27898 hybrid_clock.cc:648] HybridClock initialized: now 1786158136885668 us; error 59 us; skew 500 ppm
I20260808 03:02:16.886119 27898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:16 dist-test-slave-6ntk krb5kdc[27728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158136, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:16.889439 27898 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:02:16.891027 27898 webserver.cc:460] Webserver started at http://127.0.205.129:35675/ using document root <none> and password file <none>
I20260808 03:02:16.891775 27898 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:16.891944 27898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:16.892285 27898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:16.895699 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "5435358e99d6454089c0e9496ef03345"
format_stamp: "Formatted at 2026-08-08 03:02:16 on dist-test-slave-6ntk"
server_key: "bd9e890b1b28d40e34161c63916ca94a"
server_key_iv: "eb07ef53d31bf5e6b91d9631e191fc67"
server_key_version: "encryptionkey@0"
I20260808 03:02:16.896548 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "5435358e99d6454089c0e9496ef03345"
format_stamp: "Formatted at 2026-08-08 03:02:16 on dist-test-slave-6ntk"
server_key: "bd9e890b1b28d40e34161c63916ca94a"
server_key_iv: "eb07ef53d31bf5e6b91d9631e191fc67"
server_key_version: "encryptionkey@0"
I20260808 03:02:16.902068 27898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260808 03:02:16.905649 27924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:16.907127 27898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:02:16.907308 27898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "5435358e99d6454089c0e9496ef03345"
format_stamp: "Formatted at 2026-08-08 03:02:16 on dist-test-slave-6ntk"
server_key: "bd9e890b1b28d40e34161c63916ca94a"
server_key_iv: "eb07ef53d31bf5e6b91d9631e191fc67"
server_key_version: "encryptionkey@0"
I20260808 03:02:16.907534 27898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:16.930529 27898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:16.935120 27898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:16.935456 27898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:16.936595 27898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:16.938091 27898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:16.938202 27898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:16.938361 27898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:16.938439 27898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:16.972677 27898 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:45035
I20260808 03:02:16.972716 28115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:45035 every 8 connection(s)
I20260808 03:02:16.974079 27898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:02:16.974489 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 27898
I20260808 03:02:16.974707 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 03:02:16.975346 822 external_mini_cluster.cc:1428] Setting key 97b4a3213102fe241e3c3649bb468360
Aug 08 03:02:16 dist-test-slave-6ntk krb5kdc[27728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158136, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:16.999861 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:16.977291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:55653 (local address 127.0.205.190:43441)
0808 03:02:16.977796 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:16.977802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:16.979357 (+ 1555us) server_negotiation.cc:407] Connection header received
0808 03:02:16.982464 (+ 3107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:16.982470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:16.982545 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:16.982648 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:16.984921 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:16.985656 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:16.988286 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:16.988691 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:16.988798 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:16.993615 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:16.993621 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:16.993625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:16.993661 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:16.996537 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:16.997363 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:16.997368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:16.997370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:16.997498 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:16.998008 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:16.998012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:16.998014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:16.998285 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:16.998463 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:16.999227 (+ 764us) server_negotiation.cc:299] Negotiation successful
0808 03:02:16.999477 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":262,"threads_started":1}
I20260808 03:02:17.000944 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:16.977933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43441 (local address 127.0.205.129:55653)
0808 03:02:16.979076 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:16.979133 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:16.982115 (+ 2982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:16.982953 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:16.982968 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:16.983825 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:16.984715 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:16.984743 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:16.985857 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:16.985864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:16.988102 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:16.988120 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:16.988945 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:16.988953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:16.989291 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:16.990152 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:02:16.990204 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:16.993332 (+ 3128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:16.996709 (+ 3377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:16.996718 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:16.996737 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:16.997211 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:16.997621 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:16.997627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:16.997630 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:16.997843 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:16.998430 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:16.998440 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:16.998818 (+ 378us) client_negotiation.cc:769] Sending connection context
0808 03:02:16.999106 (+ 288us) client_negotiation.cc:240] Negotiation successful
0808 03:02:16.999696 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":531,"threads_started":1}
I20260808 03:02:17.002684 28116 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43441
I20260808 03:02:17.003021 28116 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:17.003768 28116 heartbeater.cc:511] Master 127.0.205.190:43441 requested a full tablet report, sending...
I20260808 03:02:17.006328 27800 ts_manager.cc:194] Registered new tserver with Master: 5435358e99d6454089c0e9496ef03345 (127.0.205.129:45035)
I20260808 03:02:17.008708 27800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:55653
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 03:02:17.051357 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43441
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:17.424995 28123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:17.425372 28123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:17.425613 28123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:17.433516 28123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:17.433679 28123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:17.433898 28123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:02:17.444393 28123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:17.445996 28123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:17.447486 28123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:17.463038 28136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:17.463243 28137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:17.463423 28140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:17.469070 28123 server_base.cc:1034] running on GCE node
I20260808 03:02:17.469827 28123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:17.471109 28123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:17.472371 28123 hybrid_clock.cc:648] HybridClock initialized: now 1786158137472325 us; error 37 us; skew 500 ppm
I20260808 03:02:17.472738 28123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:17 dist-test-slave-6ntk krb5kdc[27728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158137, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:17.476675 28123 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:02:17.478638 28123 webserver.cc:460] Webserver started at http://127.0.205.130:37935/ using document root <none> and password file <none>
I20260808 03:02:17.479368 28123 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:17.479477 28123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:17.479768 28123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:17.482949 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "ef7eced651bf4d82aea11e3a84771d5b"
format_stamp: "Formatted at 2026-08-08 03:02:17 on dist-test-slave-6ntk"
server_key: "2646a753410d7836ae2582356d005ee4"
server_key_iv: "ba2b21113938d9e0ea5ec46bf024fb6f"
server_key_version: "encryptionkey@0"
I20260808 03:02:17.483705 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "ef7eced651bf4d82aea11e3a84771d5b"
format_stamp: "Formatted at 2026-08-08 03:02:17 on dist-test-slave-6ntk"
server_key: "2646a753410d7836ae2582356d005ee4"
server_key_iv: "ba2b21113938d9e0ea5ec46bf024fb6f"
server_key_version: "encryptionkey@0"
I20260808 03:02:17.489085 28123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:02:17.493193 28148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:17.494765 28123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:02:17.494905 28123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "ef7eced651bf4d82aea11e3a84771d5b"
format_stamp: "Formatted at 2026-08-08 03:02:17 on dist-test-slave-6ntk"
server_key: "2646a753410d7836ae2582356d005ee4"
server_key_iv: "ba2b21113938d9e0ea5ec46bf024fb6f"
server_key_version: "encryptionkey@0"
I20260808 03:02:17.495072 28123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:17.511683 28123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:17.516227 28123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:17.516508 28123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:17.517495 28123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:17.518980 28123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:17.519061 28123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:17.519157 28123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:17.519215 28123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:17.554409 28123 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:33771
I20260808 03:02:17.554502 28311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:33771 every 8 connection(s)
I20260808 03:02:17.555859 28123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:02:17.566274 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 28123
I20260808 03:02:17.566506 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 03:02:17.566991 822 external_mini_cluster.cc:1428] Setting key 0c6c8d796b27521c840fa81f472a74ce
Aug 08 03:02:17 dist-test-slave-6ntk krb5kdc[27728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158137, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:17.580077 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:17.558541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:43927 (local address 127.0.205.190:43441)
0808 03:02:17.559018 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:17.559023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:17.560076 (+ 1053us) server_negotiation.cc:407] Connection header received
0808 03:02:17.562370 (+ 2294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:17.562376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:17.562480 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:17.562609 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:17.564319 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:17.565045 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:17.567430 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:17.568009 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:17.568151 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:17.572190 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:17.572200 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:17.572207 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:17.572254 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:17.576236 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:17.577100 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:17.577109 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:17.577115 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:17.577288 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:17.577798 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:17.577805 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:17.577810 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:17.578260 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:17.578468 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:17.579293 (+ 825us) server_negotiation.cc:299] Negotiation successful
0808 03:02:17.579580 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":256,"threads_started":1}
I20260808 03:02:17.580806 28317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:17.559021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43441 (local address 127.0.205.130:43927)
0808 03:02:17.559904 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:17.559947 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:17.562033 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:17.562887 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:17.562897 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:17.563480 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:17.564110 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:17.564129 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:17.565231 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:17.565236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:17.567184 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:17.567204 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:17.568289 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:17.568300 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:17.568600 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:17.569495 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:02:17.569522 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:17.571907 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:17.576428 (+ 4521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:17.576437 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:17.576453 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:17.576946 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:17.577442 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:17.577446 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:17.577448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:17.577664 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:17.578445 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:17.578476 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:17.578862 (+ 386us) client_negotiation.cc:769] Sending connection context
0808 03:02:17.579136 (+ 274us) client_negotiation.cc:240] Negotiation successful
0808 03:02:17.579598 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":317,"threads_started":1}
I20260808 03:02:17.582556 28313 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43441
I20260808 03:02:17.582909 28313 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:17.583686 28313 heartbeater.cc:511] Master 127.0.205.190:43441 requested a full tablet report, sending...
I20260808 03:02:17.585268 27800 ts_manager.cc:194] Registered new tserver with Master: ef7eced651bf4d82aea11e3a84771d5b (127.0.205.130:33771)
I20260808 03:02:17.586627 27800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:43927
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 03:02:17.645155 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43441
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:18.003448 28324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:18.003736 28324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:18.003830 28324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:18.011193 28324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:18.011436 28324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:18.011626 28324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:02:18.012204 28116 heartbeater.cc:503] Master 127.0.205.190:43441 was elected leader, sending a full tablet report...
I20260808 03:02:18.022697 28324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:43441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:18.023957 28324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:18.025197 28324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:18.038825 28331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:18.038780 28336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:18.038915 28333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:18.039676 28324 server_base.cc:1034] running on GCE node
I20260808 03:02:18.040311 28324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:18.041378 28324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:18.042585 28324 hybrid_clock.cc:648] HybridClock initialized: now 1786158138042548 us; error 36 us; skew 500 ppm
I20260808 03:02:18.042929 28324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:18 dist-test-slave-6ntk krb5kdc[27728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158138, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:18.046208 28324 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:02:18.047744 28324 webserver.cc:460] Webserver started at http://127.0.205.131:33657/ using document root <none> and password file <none>
I20260808 03:02:18.048465 28324 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:18.048615 28324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:18.048945 28324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:18.051941 28324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "6f6d3e0c670a45ebb399cebebc884819"
format_stamp: "Formatted at 2026-08-08 03:02:18 on dist-test-slave-6ntk"
server_key: "a09ce81c55148b0ef3f2f13ed987ff31"
server_key_iv: "590b436eef5825c9c4e8f09c7b6b3f26"
server_key_version: "encryptionkey@0"
I20260808 03:02:18.052848 28324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "6f6d3e0c670a45ebb399cebebc884819"
format_stamp: "Formatted at 2026-08-08 03:02:18 on dist-test-slave-6ntk"
server_key: "a09ce81c55148b0ef3f2f13ed987ff31"
server_key_iv: "590b436eef5825c9c4e8f09c7b6b3f26"
server_key_version: "encryptionkey@0"
I20260808 03:02:18.058413 28324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 03:02:18.062222 28345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:18.063678 28324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:18.063865 28324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "6f6d3e0c670a45ebb399cebebc884819"
format_stamp: "Formatted at 2026-08-08 03:02:18 on dist-test-slave-6ntk"
server_key: "a09ce81c55148b0ef3f2f13ed987ff31"
server_key_iv: "590b436eef5825c9c4e8f09c7b6b3f26"
server_key_version: "encryptionkey@0"
I20260808 03:02:18.064090 28324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:18.080212 28324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:18.084798 28324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:18.085136 28324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:18.086252 28324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:18.087757 28324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:18.087865 28324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:18.088028 28324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:18.088109 28324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:18.124806 28324 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:39281
I20260808 03:02:18.124859 28531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:39281 every 8 connection(s)
I20260808 03:02:18.126250 28324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:02:18.128463 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 28324
I20260808 03:02:18.128666 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 03:02:18.129280 822 external_mini_cluster.cc:1428] Setting key 8ab6c2367f3ea124d9d8db14f3add51b
Aug 08 03:02:18 dist-test-slave-6ntk krb5kdc[27728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158138, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:18.148136 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.129075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:51813 (local address 127.0.205.190:43441)
0808 03:02:18.129714 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.129722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.130867 (+ 1145us) server_negotiation.cc:407] Connection header received
0808 03:02:18.132778 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.132783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.132850 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.132942 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:18.134642 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.135516 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.137051 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.137468 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.137580 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.141382 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:18.141388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:18.141392 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:18.141432 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:18.144830 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:18.145607 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:18.145613 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:18.145616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:18.145752 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:18.146259 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:18.146266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:18.146275 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:18.146553 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:18.146731 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.147514 (+ 783us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.147745 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":364,"threads_started":1}
I20260808 03:02:18.149197 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.129704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:43441 (local address 127.0.205.131:51813)
0808 03:02:18.130689 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:18.130745 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:18.132520 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:18.133165 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:18.133173 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:18.133794 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:18.134471 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.134489 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.135662 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.135665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.136901 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.136913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.137671 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.137675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.137943 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.138699 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:02:18.138724 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:18.141031 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:18.144980 (+ 3949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:18.144990 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:18.145014 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:18.145470 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:18.145893 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:18.145912 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:18.145915 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:18.146130 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:18.146708 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:18.146716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:18.147091 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 03:02:18.147407 (+ 316us) client_negotiation.cc:240] Negotiation successful
0808 03:02:18.147997 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":410,"threads_started":1}
I20260808 03:02:18.150775 28532 heartbeater.cc:347] Connected to a master server at 127.0.205.190:43441
I20260808 03:02:18.151065 28532 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:18.151870 28532 heartbeater.cc:511] Master 127.0.205.190:43441 requested a full tablet report, sending...
I20260808 03:02:18.153491 27800 ts_manager.cc:194] Registered new tserver with Master: 6f6d3e0c670a45ebb399cebebc884819 (127.0.205.131:39281)
I20260808 03:02:18.154454 27800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:51813
I20260808 03:02:18.157371 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:02:18.176297 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.162040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38726 (local address 127.0.205.190:43441)
0808 03:02:18.162224 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.162229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.162655 (+ 426us) server_negotiation.cc:407] Connection header received
0808 03:02:18.162767 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.162771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.162835 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.162924 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:18.164346 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.165175 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.166664 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.167113 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.167261 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.170390 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:18.170396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:18.170400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:18.170438 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:18.173401 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:18.174408 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:18.174412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:18.174415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:18.174549 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:18.174963 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:18.174966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:18.174968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:18.175298 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:18.175556 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.175895 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.176053 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:02:18.180678 27800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:02:18.188890 27800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:02:18.236030 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.203946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55676 (local address 127.0.205.131:39281)
0808 03:02:18.227509 (+ 23563us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.227517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.227551 (+ 34us) server_negotiation.cc:407] Connection header received
0808 03:02:18.227660 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.227669 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.227914 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.228826 (+ 912us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:18.229110 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.230865 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.233338 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.234808 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.234957 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.235201 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.235459 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.235649 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23355,"thread_start_us":23152,"threads_started":1}
I20260808 03:02:18.236024 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.203345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:33771 (local address 127.0.0.1:49360)
0808 03:02:18.223162 (+ 19817us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:18.223225 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:18.223488 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:18.226709 (+ 3221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:18.226718 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:18.226751 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:18.227305 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.227322 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.229630 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.229637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.232687 (+ 3050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.232712 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.234676 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.234684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.234920 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.234989 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 03:02:18.235158 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 03:02:18.235446 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19480,"mutex_wait_us":81,"thread_start_us":21294,"threads_started":1}
I20260808 03:02:18.236068 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.203826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:39281 (local address 127.0.0.1:55676)
0808 03:02:18.224032 (+ 20206us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:18.224057 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:18.224199 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:18.228175 (+ 3976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:18.228181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:18.228210 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:18.228602 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.228615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.231040 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.231046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.233152 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.233175 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.235042 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.235049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.235221 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.235289 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 03:02:18.235392 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 03:02:18.235530 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20029,"mutex_wait_us":63,"thread_start_us":20462,"threads_started":1}
I20260808 03:02:18.236936 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.207051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.0.205.129:45035)
0808 03:02:18.226810 (+ 19759us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.226823 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.226862 (+ 39us) server_negotiation.cc:407] Connection header received
0808 03:02:18.227003 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.227015 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.227277 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.227489 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:18.228279 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.229820 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.234018 (+ 4198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.235491 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.235623 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.235851 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.236444 (+ 593us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.236606 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19482,"thread_start_us":19283,"threads_started":1}
I20260808 03:02:18.237223 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.203456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49360 (local address 127.0.205.130:33771)
0808 03:02:18.225783 (+ 22327us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.225798 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.225842 (+ 44us) server_negotiation.cc:407] Connection header received
0808 03:02:18.226052 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.226067 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.226428 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.226686 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:18.227536 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.229445 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.232894 (+ 3449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.234445 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.234598 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.234801 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.236640 (+ 1839us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.236849 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22066,"thread_start_us":21886,"threads_started":1}
I20260808 03:02:18.237257 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.206608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:45035 (local address 127.0.0.1:46960)
0808 03:02:18.224938 (+ 18330us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:18.224957 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:18.225071 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:18.227747 (+ 2676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:18.227751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:18.227777 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:18.228097 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.228106 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.230002 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.230008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.233803 (+ 3795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.233821 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.236042 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.236053 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.236244 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.236279 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 03:02:18.236377 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 03:02:18.236503 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18208,"mutex_wait_us":870,"thread_start_us":16046,"threads_started":1}
I20260808 03:02:18.240856 28421 tablet_service.cc:1467] Processing CreateTablet for tablet fa9a078fb82b4841a553d11e65f29884 (DEFAULT_TABLE table=test-table [id=2995cc59afe34db7a2fc7899e6b577ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:18.241072 28001 tablet_service.cc:1467] Processing CreateTablet for tablet fa9a078fb82b4841a553d11e65f29884 (DEFAULT_TABLE table=test-table [id=2995cc59afe34db7a2fc7899e6b577ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:18.241073 28229 tablet_service.cc:1467] Processing CreateTablet for tablet fa9a078fb82b4841a553d11e65f29884 (DEFAULT_TABLE table=test-table [id=2995cc59afe34db7a2fc7899e6b577ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:18.242388 28421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9a078fb82b4841a553d11e65f29884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:18.242524 28229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9a078fb82b4841a553d11e65f29884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:18.242553 28001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa9a078fb82b4841a553d11e65f29884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:18.256004 28562 tablet_bootstrap.cc:492] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: Bootstrap starting.
I20260808 03:02:18.260311 28562 tablet_bootstrap.cc:654] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:18.262095 28562 log.cc:824] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:18.263348 28563 tablet_bootstrap.cc:492] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: Bootstrap starting.
I20260808 03:02:18.265529 28562 tablet_bootstrap.cc:492] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: No bootstrap required, opened a new log
I20260808 03:02:18.265837 28562 ts_tablet_manager.cc:1397] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260808 03:02:18.267733 28563 tablet_bootstrap.cc:654] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:18.268761 28563 log.cc:824] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:18.268956 28565 tablet_bootstrap.cc:492] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: Bootstrap starting.
I20260808 03:02:18.273101 28563 tablet_bootstrap.cc:492] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: No bootstrap required, opened a new log
I20260808 03:02:18.273461 28563 ts_tablet_manager.cc:1397] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260808 03:02:18.273633 28565 tablet_bootstrap.cc:654] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:18.275079 28565 log.cc:824] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:18.276342 28562 raft_consensus.cc:348] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 } } peers { permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 } } peers { permanent_uuid: "5435358e99d6454089c0e9496ef03345" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 45035 } }
I20260808 03:02:18.276939 28562 raft_consensus.cc:374] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:18.277086 28562 raft_consensus.cc:729] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f6d3e0c670a45ebb399cebebc884819, State: Initialized, Role: FOLLOWER
I20260808 03:02:18.277851 28562 consensus_queue.cc:260] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 } } peers { permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 } } peers { permanent_uuid: "5435358e99d6454089c0e9496ef03345" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 45035 } }
I20260808 03:02:18.279764 28565 tablet_bootstrap.cc:492] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: No bootstrap required, opened a new log
I20260808 03:02:18.279863 28532 heartbeater.cc:503] Master 127.0.205.190:43441 was elected leader, sending a full tablet report...
I20260808 03:02:18.280131 28565 ts_tablet_manager.cc:1397] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.000s
I20260808 03:02:18.280579 28562 ts_tablet_manager.cc:1428] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
I20260808 03:02:18.287654 28563 raft_consensus.cc:348] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 } } peers { permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 } } peers { permanent_uuid: "5435358e99d6454089c0e9496ef03345" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 45035 } }
I20260808 03:02:18.288089 28563 raft_consensus.cc:374] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:18.288234 28563 raft_consensus.cc:729] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef7eced651bf4d82aea11e3a84771d5b, State: Initialized, Role: FOLLOWER
I20260808 03:02:18.288928 28563 consensus_queue.cc:260] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 } } peers { permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 } } peers { permanent_uuid: "5435358e99d6454089c0e9496ef03345" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 45035 } }
I20260808 03:02:18.289278 28565 raft_consensus.cc:348] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 } } peers { permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 } } peers { permanent_uuid: "5435358e99d6454089c0e9496ef03345" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 45035 } }
I20260808 03:02:18.289731 28565 raft_consensus.cc:374] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:18.289837 28565 raft_consensus.cc:729] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5435358e99d6454089c0e9496ef03345, State: Initialized, Role: FOLLOWER
I20260808 03:02:18.290169 28313 heartbeater.cc:503] Master 127.0.205.190:43441 was elected leader, sending a full tablet report...
I20260808 03:02:18.290512 28565 consensus_queue.cc:260] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 } } peers { permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 } } peers { permanent_uuid: "5435358e99d6454089c0e9496ef03345" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 45035 } }
I20260808 03:02:18.290922 28563 ts_tablet_manager.cc:1428] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: Time spent starting tablet: real 0.017s user 0.008s sys 0.005s
I20260808 03:02:18.292413 28565 ts_tablet_manager.cc:1428] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
W20260808 03:02:18.307925 28314 tablet.cc:2369] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:02:18.320094 28574 raft_consensus.cc:482] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:02:18.320398 28574 raft_consensus.cc:504] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 } } peers { permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 } } peers { permanent_uuid: "5435358e99d6454089c0e9496ef03345" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 45035 } }
I20260808 03:02:18.322276 28574 leader_election.cc:288] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef7eced651bf4d82aea11e3a84771d5b (127.0.205.130:33771), 6f6d3e0c670a45ebb399cebebc884819 (127.0.205.131:39281)
I20260808 03:02:18.330263 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.322916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:46177 (local address 127.0.205.131:39281)
0808 03:02:18.323368 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.323375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.324000 (+ 625us) server_negotiation.cc:407] Connection header received
0808 03:02:18.324259 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.324268 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.324379 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.324481 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:18.325177 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.326248 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.328224 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.329145 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.329350 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.329468 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.329754 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.329954 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254}
I20260808 03:02:18.330346 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.323066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:45055 (local address 127.0.205.130:33771)
0808 03:02:18.323478 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.323483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.324130 (+ 647us) server_negotiation.cc:407] Connection header received
0808 03:02:18.324468 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.324473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.324557 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.324668 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:18.325486 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.326335 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.328399 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.329177 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.329383 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.329462 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.329942 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.330105 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254}
I20260808 03:02:18.330679 28579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.322553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:39281 (local address 127.0.205.129:46177)
0808 03:02:18.323869 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:18.323887 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:18.324027 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:18.324499 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:18.324504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:18.324539 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:18.324998 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.325013 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.326397 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.326405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.328032 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.328044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.329407 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.329413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.329578 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.329606 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 03:02:18.329683 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 03:02:18.329802 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1178,"mutex_wait_us":44,"thread_start_us":1077,"threads_started":1}
I20260808 03:02:18.331282 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.322617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:33771 (local address 127.0.205.129:45055)
0808 03:02:18.323992 (+ 1375us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:18.324013 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:18.324230 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:18.324728 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:18.324731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:18.324758 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:18.325282 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.325301 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.326500 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.326504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.328224 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:18.328240 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.329463 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.329471 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:18.329655 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.329737 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 03:02:18.329851 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 03:02:18.330832 (+ 981us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1229,"mutex_wait_us":49,"thread_start_us":569,"threads_started":1}
I20260808 03:02:18.332185 28453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9a078fb82b4841a553d11e65f29884" candidate_uuid: "5435358e99d6454089c0e9496ef03345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f6d3e0c670a45ebb399cebebc884819" is_pre_election: true
I20260808 03:02:18.332338 28254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9a078fb82b4841a553d11e65f29884" candidate_uuid: "5435358e99d6454089c0e9496ef03345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef7eced651bf4d82aea11e3a84771d5b" is_pre_election: true
I20260808 03:02:18.332829 28254 raft_consensus.cc:2454] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5435358e99d6454089c0e9496ef03345 in term 0.
I20260808 03:02:18.332831 28453 raft_consensus.cc:2454] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5435358e99d6454089c0e9496ef03345 in term 0.
I20260808 03:02:18.333613 27926 leader_election.cc:302] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5435358e99d6454089c0e9496ef03345, 6f6d3e0c670a45ebb399cebebc884819; no voters:
I20260808 03:02:18.334177 28574 raft_consensus.cc:2781] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:02:18.334353 28574 raft_consensus.cc:482] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:02:18.334491 28574 raft_consensus.cc:3037] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:18.336853 28574 raft_consensus.cc:504] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 } } peers { permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 } } peers { permanent_uuid: "5435358e99d6454089c0e9496ef03345" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 45035 } }
I20260808 03:02:18.337579 28574 leader_election.cc:288] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [CANDIDATE]: Term 1 election: Requested vote from peers ef7eced651bf4d82aea11e3a84771d5b (127.0.205.130:33771), 6f6d3e0c670a45ebb399cebebc884819 (127.0.205.131:39281)
I20260808 03:02:18.338141 28453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9a078fb82b4841a553d11e65f29884" candidate_uuid: "5435358e99d6454089c0e9496ef03345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f6d3e0c670a45ebb399cebebc884819"
I20260808 03:02:18.338173 28254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa9a078fb82b4841a553d11e65f29884" candidate_uuid: "5435358e99d6454089c0e9496ef03345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef7eced651bf4d82aea11e3a84771d5b"
I20260808 03:02:18.338375 28453 raft_consensus.cc:3037] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:18.338420 28254 raft_consensus.cc:3037] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:18.340708 28453 raft_consensus.cc:2454] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5435358e99d6454089c0e9496ef03345 in term 1.
I20260808 03:02:18.340965 28254 raft_consensus.cc:2454] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5435358e99d6454089c0e9496ef03345 in term 1.
I20260808 03:02:18.341223 27926 leader_election.cc:302] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5435358e99d6454089c0e9496ef03345, 6f6d3e0c670a45ebb399cebebc884819; no voters:
I20260808 03:02:18.341572 28574 raft_consensus.cc:2781] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:18.342131 28574 raft_consensus.cc:686] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 1 LEADER]: Becoming Leader. State: Replica: 5435358e99d6454089c0e9496ef03345, State: Running, Role: LEADER
I20260808 03:02:18.342664 28574 consensus_queue.cc:237] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 } } peers { permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 } } peers { permanent_uuid: "5435358e99d6454089c0e9496ef03345" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 45035 } }
I20260808 03:02:18.347836 27799 catalog_manager.cc:5495] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5435358e99d6454089c0e9496ef03345 (127.0.205.129). New cstate: current_term: 1 leader_uuid: "5435358e99d6454089c0e9496ef03345" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5435358e99d6454089c0e9496ef03345" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 45035 } health_report { overall_health: HEALTHY } } }
I20260808 03:02:18.370187 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.364147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46972 (local address 127.0.205.129:45035)
0808 03:02:18.364385 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.364391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.364413 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:02:18.364516 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.364519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.364617 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.364783 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:02:18.365200 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.366004 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.367562 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.368080 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.368218 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.368942 (+ 724us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:02:18.369504 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:02:18.369721 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.369850 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.369932 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 03:02:18.378257 28534 tablet.cc:2369] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:02:18.379213 28254 raft_consensus.cc:1261] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b [term 1 FOLLOWER]: Refusing update from remote peer 5435358e99d6454089c0e9496ef03345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:18.379251 28453 raft_consensus.cc:1261] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819 [term 1 FOLLOWER]: Refusing update from remote peer 5435358e99d6454089c0e9496ef03345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:18.380327 28580 consensus_queue.cc:1035] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef7eced651bf4d82aea11e3a84771d5b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 33771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:18.380738 28574 consensus_queue.cc:1035] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f6d3e0c670a45ebb399cebebc884819" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:18.393837 28584 mvcc.cc:205] Tried to move back new op lower bound from 7316103734790074368 to 7316103734655291392. Current Snapshot: MvccSnapshot[applied={T|T < 7316103734790074368}]
I20260808 03:02:18.410633 27799 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:18.410979 27799 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:18.415478 27799 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87710e25bb724cb688f236f268821435: Sending DeleteTablet for 3 replicas of tablet fa9a078fb82b4841a553d11e65f29884
I20260808 03:02:18.416966 28001 tablet_service.cc:1514] Processing DeleteTablet for tablet fa9a078fb82b4841a553d11e65f29884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:18 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:46960
I20260808 03:02:18.417025 28421 tablet_service.cc:1514] Processing DeleteTablet for tablet fa9a078fb82b4841a553d11e65f29884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:18 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:55676
I20260808 03:02:18.417022 28229 tablet_service.cc:1514] Processing DeleteTablet for tablet fa9a078fb82b4841a553d11e65f29884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:18 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:49360
I20260808 03:02:18.424628 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 27898
I20260808 03:02:18.429437 28598 tablet_replica.cc:330] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: stopping tablet replica
I20260808 03:02:18.430186 28598 raft_consensus.cc:2229] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:02:18.431079 28598 raft_consensus.cc:2258] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:18.431559 28601 tablet_replica.cc:330] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: stopping tablet replica
I20260808 03:02:18.432397 28601 raft_consensus.cc:2229] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:18.433228 28601 raft_consensus.cc:2258] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:18.433707 28598 ts_tablet_manager.cc:1905] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:18.434535 28602 tablet_replica.cc:330] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: stopping tablet replica
I20260808 03:02:18.435251 28602 raft_consensus.cc:2229] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:18.436040 28602 raft_consensus.cc:2258] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:18.438687 28601 ts_tablet_manager.cc:1905] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:18.438952 28602 ts_tablet_manager.cc:1905] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:18.446931 28602 ts_tablet_manager.cc:1918] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:18.447247 28602 log.cc:1196] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/fa9a078fb82b4841a553d11e65f29884
I20260808 03:02:18.447397 28598 ts_tablet_manager.cc:1918] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:18.447718 28598 log.cc:1196] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/fa9a078fb82b4841a553d11e65f29884
I20260808 03:02:18.448004 28602 ts_tablet_manager.cc:1939] T fa9a078fb82b4841a553d11e65f29884 P 6f6d3e0c670a45ebb399cebebc884819: Deleting consensus metadata
I20260808 03:02:18.448406 28598 ts_tablet_manager.cc:1939] T fa9a078fb82b4841a553d11e65f29884 P 5435358e99d6454089c0e9496ef03345: Deleting consensus metadata
I20260808 03:02:18.450357 27780 catalog_manager.cc:4841] TS 6f6d3e0c670a45ebb399cebebc884819 (127.0.205.131:39281): tablet fa9a078fb82b4841a553d11e65f29884 (table test-table [id=2995cc59afe34db7a2fc7899e6b577ac]) successfully deleted
I20260808 03:02:18.450439 27781 catalog_manager.cc:4841] TS 5435358e99d6454089c0e9496ef03345 (127.0.205.129:45035): tablet fa9a078fb82b4841a553d11e65f29884 (table test-table [id=2995cc59afe34db7a2fc7899e6b577ac]) successfully deleted
I20260808 03:02:18.451236 28601 ts_tablet_manager.cc:1918] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:18.451433 28601 log.cc:1196] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/fa9a078fb82b4841a553d11e65f29884
I20260808 03:02:18.451967 28601 ts_tablet_manager.cc:1939] T fa9a078fb82b4841a553d11e65f29884 P ef7eced651bf4d82aea11e3a84771d5b: Deleting consensus metadata
I20260808 03:02:18.454167 27778 catalog_manager.cc:4841] TS ef7eced651bf4d82aea11e3a84771d5b (127.0.205.130:33771): tablet fa9a078fb82b4841a553d11e65f29884 (table test-table [id=2995cc59afe34db7a2fc7899e6b577ac]) successfully deleted
Aug 08 03:02:18 dist-test-slave-6ntk krb5kdc[27728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 03:02:18.465648 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.426703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46978 (local address 127.0.205.129:45035)
0808 03:02:18.427031 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.427042 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.437183 (+ 10141us) server_negotiation.cc:407] Connection header received
0808 03:02:18.437536 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.437543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.437645 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.437736 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:18.440053 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.441361 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.443524 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.444366 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.444495 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.457676 (+ 13181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:18.457694 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:18.457711 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:18.457762 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:18.462082 (+ 4320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:18.462995 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:18.463004 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:18.463007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:18.463240 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:18.463745 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:18.463749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:18.463752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:18.464214 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:18.464573 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.464962 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.465256 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260808 03:02:18.466500 28106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:18.575814 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 27898
I20260808 03:02:18.602134 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 28123
Aug 08 03:02:18 dist-test-slave-6ntk krb5kdc[27728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 03:02:18.617506 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.602978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49366 (local address 127.0.205.130:33771)
0808 03:02:18.603153 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.603158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.603176 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:02:18.603348 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.603352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.603429 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.603519 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:18.604799 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.605548 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.606911 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.607292 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.607394 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.611986 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:18.612004 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:18.612022 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:18.612057 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:18.615006 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:18.615698 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:18.615706 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:18.615708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:18.615838 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:18.616173 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:18.616176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:18.616178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:18.616564 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:18.616794 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.617065 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.617240 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 03:02:18.618319 28307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:18.712124 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 28123
I20260808 03:02:18.738227 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 28324
Aug 08 03:02:18 dist-test-slave-6ntk krb5kdc[27728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 03:02:18.754504 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:18.739129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55690 (local address 127.0.205.131:39281)
0808 03:02:18.739328 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:18.739333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:18.739352 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:02:18.739522 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:18.739527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:18.739605 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:18.739692 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:18.741091 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.741816 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.743139 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:18.743534 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:18.743633 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:18.748517 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:18.748535 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:18.748552 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:18.748591 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:18.751435 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:18.752192 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:18.752200 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:18.752203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:18.752354 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:18.752776 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:18.752779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:18.752781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:18.753156 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:18.753548 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:18.753893 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 03:02:18.754147 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 03:02:18.755227 28526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:18.847716 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 28324
I20260808 03:02:18.875973 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 27755
I20260808 03:02:18.877214 27843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:18.992789 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 27755
2026-08-08T03:02:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4849 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T03:02:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:19Z Disabled control of system clock
I20260808 03:02:19.097046 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:44211
--webserver_interface=127.0.205.190
--webserver_port=0
--builtin_ntp_servers=127.0.205.148:41237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:44211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:02:19.470050 28616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:19.470383 28616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:19.470477 28616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:19.478276 28616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:19.478416 28616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:19.478474 28616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:19.478541 28616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:19.478595 28616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:19.488374 28616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:41237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:44211
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:44211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:19.489740 28616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:19.491683 28616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:19.504351 28626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:19.504453 28623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:19.504453 28624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:19.505335 28616 server_base.cc:1034] running on GCE node
I20260808 03:02:19.506371 28616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:19.507889 28616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:19.509135 28616 hybrid_clock.cc:648] HybridClock initialized: now 1786158139509081 us; error 49 us; skew 500 ppm
I20260808 03:02:19.509526 28616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:19.512215 28616 webserver.cc:460] Webserver started at http://127.0.205.190:45539/ using document root <none> and password file <none>
I20260808 03:02:19.512987 28616 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:19.513129 28616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:19.513491 28616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:19.516633 28616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "81eec70a991e43e588916e4322824a20"
format_stamp: "Formatted at 2026-08-08 03:02:19 on dist-test-slave-6ntk"
server_key: "a206504bafc156e297685af55b5d519f"
server_key_iv: "78e81c5a73282c349285950547ee3090"
server_key_version: "encryptionkey@0"
I20260808 03:02:19.517484 28616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "81eec70a991e43e588916e4322824a20"
format_stamp: "Formatted at 2026-08-08 03:02:19 on dist-test-slave-6ntk"
server_key: "a206504bafc156e297685af55b5d519f"
server_key_iv: "78e81c5a73282c349285950547ee3090"
server_key_version: "encryptionkey@0"
I20260808 03:02:19.523268 28616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 03:02:19.527031 28635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:19.528455 28616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:02:19.528657 28616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "81eec70a991e43e588916e4322824a20"
format_stamp: "Formatted at 2026-08-08 03:02:19 on dist-test-slave-6ntk"
server_key: "a206504bafc156e297685af55b5d519f"
server_key_iv: "78e81c5a73282c349285950547ee3090"
server_key_version: "encryptionkey@0"
I20260808 03:02:19.528893 28616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:19.544160 28616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:19.545054 28616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:19.545383 28616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:19.564845 28616 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:44211
I20260808 03:02:19.564868 28713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:44211 every 8 connection(s)
I20260808 03:02:19.566345 28616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:19.570016 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 28616
I20260808 03:02:19.570209 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:19.570406 28714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:19.570783 822 external_mini_cluster.cc:1428] Setting key 882c7a6185eb7cc8bd4270df71777bb5
I20260808 03:02:19.586892 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:19.572616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51782 (local address 127.0.205.190:44211)
0808 03:02:19.573735 (+ 1119us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:19.573753 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:19.573810 (+ 57us) server_negotiation.cc:407] Connection header received
0808 03:02:19.575672 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:19.575702 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:19.576360 (+ 658us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:19.576843 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:19.577490 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:19.579654 (+ 2164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:19.581120 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:19.582252 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:19.582472 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:19.583688 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:19.583700 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:19.583725 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:19.584571 (+ 846us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:19.584694 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:19.584722 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:19.584838 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:19.585096 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 03:02:19.585498 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":387,"threads_started":1}
I20260808 03:02:19.588258 28714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20: Bootstrap starting.
I20260808 03:02:19.591739 28714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:19.592869 28714 log.cc:824] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:19.596273 28714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20: No bootstrap required, opened a new log
I20260808 03:02:19.604321 28714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81eec70a991e43e588916e4322824a20" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44211 } }
I20260808 03:02:19.604780 28714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:19.604919 28714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81eec70a991e43e588916e4322824a20, State: Initialized, Role: FOLLOWER
I20260808 03:02:19.605509 28714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81eec70a991e43e588916e4322824a20" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44211 } }
I20260808 03:02:19.605767 28714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:19.605981 28714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:19.606187 28714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:19.608733 28714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81eec70a991e43e588916e4322824a20" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44211 } }
I20260808 03:02:19.609297 28714 leader_election.cc:302] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81eec70a991e43e588916e4322824a20; no voters:
I20260808 03:02:19.610002 28714 leader_election.cc:288] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:19.610150 28725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:19.610908 28725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [term 1 LEADER]: Becoming Leader. State: Replica: 81eec70a991e43e588916e4322824a20, State: Running, Role: LEADER
I20260808 03:02:19.611503 28725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81eec70a991e43e588916e4322824a20" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44211 } }
I20260808 03:02:19.612305 28714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:19.614810 28726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81eec70a991e43e588916e4322824a20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81eec70a991e43e588916e4322824a20" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44211 } } }
I20260808 03:02:19.615208 28727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81eec70a991e43e588916e4322824a20. Latest consensus state: current_term: 1 leader_uuid: "81eec70a991e43e588916e4322824a20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81eec70a991e43e588916e4322824a20" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44211 } } }
I20260808 03:02:19.615486 28726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:19.615541 28727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:19.617676 28734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:19.624415 28734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:19.634022 28734 catalog_manager.cc:1347] Generated new cluster ID: 7a1f7cf7bcfc44a391fba35537d95cca
I20260808 03:02:19.634187 28734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:19.647598 28734 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:19.649039 28734 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:19.661790 28734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20: Generated new TSK 0
I20260808 03:02:19.662803 28734 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:19.670644 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44211
--builtin_ntp_servers=127.0.205.148:41237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:02:20.082561 28753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:20.082892 28753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:20.082986 28753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:20.091423 28753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:20.091552 28753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:20.091665 28753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:02:20.101511 28753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:41237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44211
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:20.103199 28753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:20.105104 28753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:20.120266 28761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:20.120222 28762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:20.120419 28764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:20.121441 28753 server_base.cc:1034] running on GCE node
I20260808 03:02:20.122215 28753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:20.123302 28753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:20.124577 28753 hybrid_clock.cc:648] HybridClock initialized: now 1786158140124525 us; error 43 us; skew 500 ppm
I20260808 03:02:20.124972 28753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:20.127589 28753 webserver.cc:460] Webserver started at http://127.0.205.129:42395/ using document root <none> and password file <none>
I20260808 03:02:20.128456 28753 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:20.128604 28753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:20.128973 28753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:20.132266 28753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "ce1cc45b961748888d41094e9b0ad03c"
format_stamp: "Formatted at 2026-08-08 03:02:20 on dist-test-slave-6ntk"
server_key: "e8a915fc407438e4a057f7dac9d7354a"
server_key_iv: "8f65a1d9906c4f41c6e8b85266ebb6d3"
server_key_version: "encryptionkey@0"
I20260808 03:02:20.133177 28753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "ce1cc45b961748888d41094e9b0ad03c"
format_stamp: "Formatted at 2026-08-08 03:02:20 on dist-test-slave-6ntk"
server_key: "e8a915fc407438e4a057f7dac9d7354a"
server_key_iv: "8f65a1d9906c4f41c6e8b85266ebb6d3"
server_key_version: "encryptionkey@0"
I20260808 03:02:20.139014 28753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 03:02:20.142830 28771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:20.144291 28753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:20.144510 28753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "ce1cc45b961748888d41094e9b0ad03c"
format_stamp: "Formatted at 2026-08-08 03:02:20 on dist-test-slave-6ntk"
server_key: "e8a915fc407438e4a057f7dac9d7354a"
server_key_iv: "8f65a1d9906c4f41c6e8b85266ebb6d3"
server_key_version: "encryptionkey@0"
I20260808 03:02:20.144773 28753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:20.158540 28753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:20.159463 28753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:20.159803 28753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:20.160909 28753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:20.162607 28753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:20.162735 28753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:20.162878 28753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:20.162971 28753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:20.201058 28753 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:35885
I20260808 03:02:20.201115 28929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:35885 every 8 connection(s)
I20260808 03:02:20.202579 28753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:02:20.209856 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 28753
I20260808 03:02:20.210139 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 03:02:20.210644 822 external_mini_cluster.cc:1428] Setting key c2833fd66a5e12ce8a7dddf0e3fd1f60
I20260808 03:02:20.213677 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44211
--builtin_ntp_servers=127.0.205.148:41237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:02:20.216565 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:20.205392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:46073 (local address 127.0.205.190:44211)
0808 03:02:20.205918 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:20.205927 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:20.207315 (+ 1388us) server_negotiation.cc:407] Connection header received
0808 03:02:20.209185 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:20.209199 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:20.209297 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:20.209426 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:20.210737 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:20.211651 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:20.213528 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:20.214026 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:20.214171 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:20.215668 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:20.215674 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:20.215677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:20.215697 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:20.215741 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:20.215753 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:20.215851 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:20.216085 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 03:02:20.216187 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":244,"threads_started":1}
I20260808 03:02:20.217662 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:20.206109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:44211 (local address 127.0.205.129:46073)
0808 03:02:20.207081 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:20.207122 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:20.208887 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:20.209709 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:20.209718 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:20.209810 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:20.210553 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:20.210573 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:20.211817 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:20.211827 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:20.213379 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:20.213394 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:20.214269 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:20.214277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:20.214532 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:20.215314 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:02:20.215317 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:20.215362 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:02:20.215448 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:20.215853 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:20.215868 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:20.215878 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 03:02:20.216072 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 03:02:20.216493 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":322,"threads_started":1}
I20260808 03:02:20.219506 28930 heartbeater.cc:347] Connected to a master server at 127.0.205.190:44211
I20260808 03:02:20.219842 28930 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:20.220654 28930 heartbeater.cc:511] Master 127.0.205.190:44211 requested a full tablet report, sending...
I20260808 03:02:20.223191 28656 ts_manager.cc:194] Registered new tserver with Master: ce1cc45b961748888d41094e9b0ad03c (127.0.205.129:35885)
I20260808 03:02:20.225414 28656 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.129:46073
W20260808 03:02:20.583191 28935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:20.583521 28935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:20.583612 28935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:20.591471 28935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:20.591593 28935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:20.591710 28935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:02:20.601428 28935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:41237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44211
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:20.602864 28935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:20.604660 28935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:20.619306 28941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:20.619277 28942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:20.619587 28944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:20.620412 28935 server_base.cc:1034] running on GCE node
I20260808 03:02:20.621060 28935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:20.622321 28935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:20.623546 28935 hybrid_clock.cc:648] HybridClock initialized: now 1786158140623476 us; error 64 us; skew 500 ppm
I20260808 03:02:20.623864 28935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:20.626616 28935 webserver.cc:460] Webserver started at http://127.0.205.130:35581/ using document root <none> and password file <none>
I20260808 03:02:20.627358 28935 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:20.627455 28935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:20.627821 28935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:20.630895 28935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "41ba7ec42e544896b1af7c407f6ebb27"
format_stamp: "Formatted at 2026-08-08 03:02:20 on dist-test-slave-6ntk"
server_key: "6799869aacfde8fd0d13e199a1c34c3f"
server_key_iv: "2e1475f69aaa884aa162df5bf05d80ff"
server_key_version: "encryptionkey@0"
I20260808 03:02:20.631672 28935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "41ba7ec42e544896b1af7c407f6ebb27"
format_stamp: "Formatted at 2026-08-08 03:02:20 on dist-test-slave-6ntk"
server_key: "6799869aacfde8fd0d13e199a1c34c3f"
server_key_iv: "2e1475f69aaa884aa162df5bf05d80ff"
server_key_version: "encryptionkey@0"
I20260808 03:02:20.637369 28935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:02:20.641222 28955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:20.642616 28935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:02:20.642814 28935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "41ba7ec42e544896b1af7c407f6ebb27"
format_stamp: "Formatted at 2026-08-08 03:02:20 on dist-test-slave-6ntk"
server_key: "6799869aacfde8fd0d13e199a1c34c3f"
server_key_iv: "2e1475f69aaa884aa162df5bf05d80ff"
server_key_version: "encryptionkey@0"
I20260808 03:02:20.643050 28935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:20.690698 28935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:20.691655 28935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:20.692006 28935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:20.693090 28935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:20.694734 28935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:20.694869 28935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:20.695015 28935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:20.695106 28935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:20.729763 28935 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:38757
I20260808 03:02:20.729823 29136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:38757 every 8 connection(s)
I20260808 03:02:20.731132 28935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:02:20.739902 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 28935
I20260808 03:02:20.740093 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 03:02:20.740607 822 external_mini_cluster.cc:1428] Setting key 4db3acb086d7c2d72739cbb38be96615
I20260808 03:02:20.743687 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44211
--builtin_ntp_servers=127.0.205.148:41237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:02:20.744771 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:20.733812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:33813 (local address 127.0.205.190:44211)
0808 03:02:20.734365 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:20.734369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:20.735481 (+ 1112us) server_negotiation.cc:407] Connection header received
0808 03:02:20.737589 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:20.737594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:20.737694 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:20.737818 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:20.738981 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:20.739826 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:20.741636 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:20.742290 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:20.742429 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:20.743840 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:20.743849 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:20.743879 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:20.743905 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:20.743965 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:20.743981 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:20.744091 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:20.744313 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 03:02:20.744414 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":268,"threads_started":1}
I20260808 03:02:20.745952 29142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:20.734395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:44211 (local address 127.0.205.130:33813)
0808 03:02:20.735283 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:20.735319 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:20.737277 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:20.738109 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:20.738118 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:20.738209 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:20.738801 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:20.738820 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:20.739992 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:20.739997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:20.741446 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:20.741461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:20.742536 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:20.742544 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:20.742768 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:20.743503 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:02:20.743506 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:20.743554 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:02:20.743633 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:20.744089 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:20.744102 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:20.744112 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 03:02:20.744303 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 03:02:20.744718 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":257,"threads_started":1}
I20260808 03:02:20.747570 29138 heartbeater.cc:347] Connected to a master server at 127.0.205.190:44211
I20260808 03:02:20.747932 29138 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:20.748803 29138 heartbeater.cc:511] Master 127.0.205.190:44211 requested a full tablet report, sending...
I20260808 03:02:20.750391 28656 ts_manager.cc:194] Registered new tserver with Master: 41ba7ec42e544896b1af7c407f6ebb27 (127.0.205.130:38757)
I20260808 03:02:20.751609 28656 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.130:33813
W20260808 03:02:21.100152 29145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:21.100488 29145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:21.100579 29145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:21.108240 29145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:21.108366 29145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:21.108487 29145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:02:21.118424 29145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:41237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:44211
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:21.119822 29145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:21.121655 29145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:21.135965 29152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:21.136046 29155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:21.135946 29151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:21.136698 29145 server_base.cc:1034] running on GCE node
I20260808 03:02:21.137411 29145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:21.138613 29145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:21.139861 29145 hybrid_clock.cc:648] HybridClock initialized: now 1786158141139786 us; error 68 us; skew 500 ppm
I20260808 03:02:21.140252 29145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:21.142829 29145 webserver.cc:460] Webserver started at http://127.0.205.131:40435/ using document root <none> and password file <none>
I20260808 03:02:21.143601 29145 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:21.143751 29145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:21.144110 29145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:21.147459 29145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "435632dda75647568ad27ca01dd6032f"
format_stamp: "Formatted at 2026-08-08 03:02:21 on dist-test-slave-6ntk"
server_key: "c9298588fa43d9571b89ff82917feab3"
server_key_iv: "6f6c93574ca7b76729cb45b8eb12f458"
server_key_version: "encryptionkey@0"
I20260808 03:02:21.148422 29145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "435632dda75647568ad27ca01dd6032f"
format_stamp: "Formatted at 2026-08-08 03:02:21 on dist-test-slave-6ntk"
server_key: "c9298588fa43d9571b89ff82917feab3"
server_key_iv: "6f6c93574ca7b76729cb45b8eb12f458"
server_key_version: "encryptionkey@0"
I20260808 03:02:21.154321 29145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260808 03:02:21.158102 29162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:21.159514 29145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:21.159696 29145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "435632dda75647568ad27ca01dd6032f"
format_stamp: "Formatted at 2026-08-08 03:02:21 on dist-test-slave-6ntk"
server_key: "c9298588fa43d9571b89ff82917feab3"
server_key_iv: "6f6c93574ca7b76729cb45b8eb12f458"
server_key_version: "encryptionkey@0"
I20260808 03:02:21.159947 29145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:21.181207 29145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:21.182245 29145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:21.182595 29145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:21.183665 29145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:21.185272 29145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:21.185413 29145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:21.185547 29145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:21.185637 29145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:21.222685 29145 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:45383
I20260808 03:02:21.222782 29337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:45383 every 8 connection(s)
I20260808 03:02:21.224121 29145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:02:21.228302 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 29145
I20260808 03:02:21.228518 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 03:02:21.228711 28930 heartbeater.cc:503] Master 127.0.205.190:44211 was elected leader, sending a full tablet report...
I20260808 03:02:21.228911 822 external_mini_cluster.cc:1428] Setting key e303afa2d069f37d31a3d5a8bb55c099
I20260808 03:02:21.237447 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.226795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:36223 (local address 127.0.205.190:44211)
0808 03:02:21.227000 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:21.227005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:21.228410 (+ 1405us) server_negotiation.cc:407] Connection header received
0808 03:02:21.230911 (+ 2501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:21.230915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:21.230981 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:21.231107 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:21.232354 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.233278 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.234893 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.235281 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.235378 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.236733 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:21.236737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:21.236741 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:21.236759 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:21.236786 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:21.236798 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:21.236865 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:21.237082 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 03:02:21.237158 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 03:02:21.238545 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.227278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:44211 (local address 127.0.205.131:36223)
0808 03:02:21.228195 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:21.228237 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:21.230604 (+ 2367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:21.231303 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:21.231312 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:21.231400 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:21.232178 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.232199 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.233416 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.233420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.234764 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.234777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.235468 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.235473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.235674 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.236442 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:02:21.236445 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:21.236484 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:02:21.236571 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:21.236897 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:21.236905 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:21.236911 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 03:02:21.237055 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 03:02:21.237351 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":270,"threads_started":1}
I20260808 03:02:21.239995 29339 heartbeater.cc:347] Connected to a master server at 127.0.205.190:44211
I20260808 03:02:21.240300 29339 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:21.241139 29339 heartbeater.cc:511] Master 127.0.205.190:44211 requested a full tablet report, sending...
I20260808 03:02:21.242673 28656 ts_manager.cc:194] Registered new tserver with Master: 435632dda75647568ad27ca01dd6032f (127.0.205.131:45383)
I20260808 03:02:21.243710 28656 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.131:36223
I20260808 03:02:21.244530 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:02:21.255035 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.249247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51796 (local address 127.0.205.190:44211)
0808 03:02:21.249419 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:21.249424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:21.249939 (+ 515us) server_negotiation.cc:407] Connection header received
0808 03:02:21.250115 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:21.250119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:21.250179 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:21.250262 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:21.250749 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.251437 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.252740 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.253142 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.253231 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.254365 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:21.254370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:21.254373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:21.254399 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:21.254427 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:21.254439 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:21.254550 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:21.254719 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 03:02:21.254764 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 03:02:21.259126 28655 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:02:21.266454 28655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:02:21.304980 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.279078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:45383 (local address 127.0.0.1:41020)
0808 03:02:21.294348 (+ 15270us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:21.294459 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:21.294719 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:21.296522 (+ 1803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:21.296530 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:21.296572 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:21.297113 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.297130 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.299308 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.299313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.301663 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.301685 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.304055 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.304059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.304206 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.304240 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 03:02:21.304406 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 03:02:21.304610 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14968,"thread_start_us":14819,"threads_started":1}
I20260808 03:02:21.305019 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.279458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41020 (local address 127.0.205.131:45383)
0808 03:02:21.293819 (+ 14361us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:21.293830 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:21.295733 (+ 1903us) server_negotiation.cc:407] Connection header received
0808 03:02:21.295846 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:21.295854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:21.296145 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:21.296357 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:21.297420 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.299143 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.301973 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.303525 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.303669 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.303957 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:21.304459 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 03:02:21.304643 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14139,"thread_start_us":13957,"threads_started":1}
I20260808 03:02:21.306442 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.282076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36278 (local address 127.0.205.129:35885)
0808 03:02:21.295648 (+ 13572us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:21.295657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:21.297655 (+ 1998us) server_negotiation.cc:407] Connection header received
0808 03:02:21.297860 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:21.297925 (+ 65us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:21.298091 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:21.298282 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:21.299223 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.301000 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.303558 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.305024 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.305168 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.305448 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:21.306018 (+ 570us) server_negotiation.cc:299] Negotiation successful
0808 03:02:21.306153 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13380,"thread_start_us":13233,"threads_started":1}
I20260808 03:02:21.306442 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.281471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:35885 (local address 127.0.0.1:36278)
0808 03:02:21.297506 (+ 16035us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:21.297527 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:21.297687 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:21.298269 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:21.298274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:21.298299 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:21.299008 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.299022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.301170 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.301176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.303313 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.303328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.305586 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.305594 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.305769 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.305800 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 03:02:21.305963 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 03:02:21.306093 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15913,"mutex_wait_us":45,"thread_start_us":13223,"threads_started":1}
I20260808 03:02:21.307263 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.279376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:38757 (local address 127.0.0.1:59056)
0808 03:02:21.294973 (+ 15597us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:21.294995 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:21.295096 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:21.297729 (+ 2633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:21.297733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:21.297767 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:21.298167 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.298180 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.300242 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.300253 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.302505 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.302521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.304919 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.304927 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.306711 (+ 1784us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.306754 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 03:02:21.306837 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 03:02:21.306968 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15463,"thread_start_us":17850,"threads_started":1}
I20260808 03:02:21.308095 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.279724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59056 (local address 127.0.205.130:38757)
0808 03:02:21.297062 (+ 17338us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:21.297072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:21.297105 (+ 33us) server_negotiation.cc:407] Connection header received
0808 03:02:21.297217 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:21.297224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:21.297493 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:21.298363 (+ 870us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:21.298614 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.300092 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.302898 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.304399 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.304536 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.304767 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:21.307548 (+ 2781us) server_negotiation.cc:299] Negotiation successful
0808 03:02:21.307744 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17163,"thread_start_us":17054,"threads_started":1}
I20260808 03:02:21.310003 29243 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9914ebd0f946f4b85d5347912b36cc (DEFAULT_TABLE table=test-table [id=6057857e575b4a5aa883413614412881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:21.311687 29243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b9914ebd0f946f4b85d5347912b36cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:21.311476 29038 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9914ebd0f946f4b85d5347912b36cc (DEFAULT_TABLE table=test-table [id=6057857e575b4a5aa883413614412881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:21.311558 28843 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9914ebd0f946f4b85d5347912b36cc (DEFAULT_TABLE table=test-table [id=6057857e575b4a5aa883413614412881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:21.312877 29038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b9914ebd0f946f4b85d5347912b36cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:21.312932 28843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b9914ebd0f946f4b85d5347912b36cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:21.325294 29379 tablet_bootstrap.cc:492] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: Bootstrap starting.
I20260808 03:02:21.329689 29379 tablet_bootstrap.cc:654] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:21.330766 29379 log.cc:824] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:21.333665 29380 tablet_bootstrap.cc:492] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: Bootstrap starting.
I20260808 03:02:21.334102 29379 tablet_bootstrap.cc:492] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: No bootstrap required, opened a new log
I20260808 03:02:21.334395 29379 ts_tablet_manager.cc:1397] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260808 03:02:21.338665 29380 tablet_bootstrap.cc:654] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:21.339197 29381 tablet_bootstrap.cc:492] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: Bootstrap starting.
I20260808 03:02:21.340001 29380 log.cc:824] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:21.342906 29380 tablet_bootstrap.cc:492] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: No bootstrap required, opened a new log
I20260808 03:02:21.343194 29380 ts_tablet_manager.cc:1397] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260808 03:02:21.343258 29381 tablet_bootstrap.cc:654] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:21.344152 29379 raft_consensus.cc:348] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 } } peers { permanent_uuid: "41ba7ec42e544896b1af7c407f6ebb27" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 38757 } } peers { permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 } }
I20260808 03:02:21.344508 29381 log.cc:824] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:21.344658 29379 raft_consensus.cc:374] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:21.344777 29379 raft_consensus.cc:729] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ba7ec42e544896b1af7c407f6ebb27, State: Initialized, Role: FOLLOWER
I20260808 03:02:21.345568 29379 consensus_queue.cc:260] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 } } peers { permanent_uuid: "41ba7ec42e544896b1af7c407f6ebb27" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 38757 } } peers { permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 } }
I20260808 03:02:21.347826 29379 ts_tablet_manager.cc:1428] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260808 03:02:21.348436 29138 heartbeater.cc:503] Master 127.0.205.190:44211 was elected leader, sending a full tablet report...
I20260808 03:02:21.352162 29380 raft_consensus.cc:348] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 } } peers { permanent_uuid: "41ba7ec42e544896b1af7c407f6ebb27" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 38757 } } peers { permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 } }
I20260808 03:02:21.352567 29380 raft_consensus.cc:374] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:21.352694 29380 raft_consensus.cc:729] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435632dda75647568ad27ca01dd6032f, State: Initialized, Role: FOLLOWER
I20260808 03:02:21.352881 29381 tablet_bootstrap.cc:492] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: No bootstrap required, opened a new log
I20260808 03:02:21.353199 29381 ts_tablet_manager.cc:1397] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260808 03:02:21.353477 29380 consensus_queue.cc:260] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 } } peers { permanent_uuid: "41ba7ec42e544896b1af7c407f6ebb27" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 38757 } } peers { permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 } }
I20260808 03:02:21.354993 29339 heartbeater.cc:503] Master 127.0.205.190:44211 was elected leader, sending a full tablet report...
I20260808 03:02:21.356851 29380 ts_tablet_manager.cc:1428] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260808 03:02:21.362159 29381 raft_consensus.cc:348] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 } } peers { permanent_uuid: "41ba7ec42e544896b1af7c407f6ebb27" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 38757 } } peers { permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 } }
I20260808 03:02:21.362608 29381 raft_consensus.cc:374] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:21.362728 29381 raft_consensus.cc:729] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1cc45b961748888d41094e9b0ad03c, State: Initialized, Role: FOLLOWER
I20260808 03:02:21.363353 29381 consensus_queue.cc:260] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 } } peers { permanent_uuid: "41ba7ec42e544896b1af7c407f6ebb27" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 38757 } } peers { permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 } }
I20260808 03:02:21.365135 29381 ts_tablet_manager.cc:1428] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260808 03:02:21.455176 28931 tablet.cc:2369] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:02:21.476048 29340 tablet.cc:2369] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:02:21.483302 29140 tablet.cc:2369] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:02:21.698827 29386 raft_consensus.cc:482] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:02:21.699177 29386 raft_consensus.cc:504] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 } } peers { permanent_uuid: "41ba7ec42e544896b1af7c407f6ebb27" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 38757 } } peers { permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 } }
I20260808 03:02:21.700973 29386 leader_election.cc:288] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 435632dda75647568ad27ca01dd6032f (127.0.205.131:45383), ce1cc45b961748888d41094e9b0ad03c (127.0.205.129:35885)
I20260808 03:02:21.708081 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.701368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:37845 (local address 127.0.205.131:45383)
0808 03:02:21.701562 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:21.701567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:21.702579 (+ 1012us) server_negotiation.cc:407] Connection header received
0808 03:02:21.702765 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:21.702770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:21.702850 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:21.702970 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:21.703584 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.704286 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.706370 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.707115 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.707303 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.707378 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:21.707704 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 03:02:21.707835 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 03:02:21.708094 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.701681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:47035 (local address 127.0.205.129:35885)
0808 03:02:21.701957 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:21.701963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:21.702404 (+ 441us) server_negotiation.cc:407] Connection header received
0808 03:02:21.702713 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:21.702720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:21.702808 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:21.702912 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:21.703555 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.704252 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.706321 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.707163 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.707345 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.707425 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:21.707718 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 03:02:21.707853 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 03:02:21.708195 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.701429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:45383 (local address 127.0.205.130:37845)
0808 03:02:21.702470 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:21.702483 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:21.702627 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:21.702988 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:21.702992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:21.703016 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:21.703423 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.703433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.704460 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.704463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.706151 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.706170 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.707327 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.707333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.707541 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.707566 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 03:02:21.707667 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 03:02:21.707813 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":233,"thread_start_us":491,"threads_started":1}
I20260808 03:02:21.708634 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.701420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:35885 (local address 127.0.205.130:47035)
0808 03:02:21.702239 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:21.702263 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:21.702488 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:21.702943 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:21.702948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:21.702972 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:21.703403 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.703413 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.704384 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.704392 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.706115 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:21.706127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.707360 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.707367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:21.707555 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.707579 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 03:02:21.707678 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 03:02:21.707842 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":98,"thread_start_us":370,"threads_started":1}
I20260808 03:02:21.709298 29276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9914ebd0f946f4b85d5347912b36cc" candidate_uuid: "41ba7ec42e544896b1af7c407f6ebb27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435632dda75647568ad27ca01dd6032f" is_pre_election: true
I20260808 03:02:21.709761 28871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9914ebd0f946f4b85d5347912b36cc" candidate_uuid: "41ba7ec42e544896b1af7c407f6ebb27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1cc45b961748888d41094e9b0ad03c" is_pre_election: true
I20260808 03:02:21.709971 29276 raft_consensus.cc:2454] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ba7ec42e544896b1af7c407f6ebb27 in term 0.
I20260808 03:02:21.710352 28871 raft_consensus.cc:2454] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ba7ec42e544896b1af7c407f6ebb27 in term 0.
I20260808 03:02:21.710654 28960 leader_election.cc:302] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ba7ec42e544896b1af7c407f6ebb27, 435632dda75647568ad27ca01dd6032f; no voters:
I20260808 03:02:21.711061 29386 raft_consensus.cc:2781] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:02:21.711216 29386 raft_consensus.cc:482] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:02:21.711331 29386 raft_consensus.cc:3037] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:21.713583 29386 raft_consensus.cc:504] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 } } peers { permanent_uuid: "41ba7ec42e544896b1af7c407f6ebb27" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 38757 } } peers { permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 } }
I20260808 03:02:21.714293 29386 leader_election.cc:288] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [CANDIDATE]: Term 1 election: Requested vote from peers 435632dda75647568ad27ca01dd6032f (127.0.205.131:45383), ce1cc45b961748888d41094e9b0ad03c (127.0.205.129:35885)
I20260808 03:02:21.714761 29276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9914ebd0f946f4b85d5347912b36cc" candidate_uuid: "41ba7ec42e544896b1af7c407f6ebb27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435632dda75647568ad27ca01dd6032f"
I20260808 03:02:21.714808 28871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9914ebd0f946f4b85d5347912b36cc" candidate_uuid: "41ba7ec42e544896b1af7c407f6ebb27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1cc45b961748888d41094e9b0ad03c"
I20260808 03:02:21.714985 29276 raft_consensus.cc:3037] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:21.715009 28871 raft_consensus.cc:3037] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:21.717202 29276 raft_consensus.cc:2454] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ba7ec42e544896b1af7c407f6ebb27 in term 1.
I20260808 03:02:21.717463 28871 raft_consensus.cc:2454] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ba7ec42e544896b1af7c407f6ebb27 in term 1.
I20260808 03:02:21.717684 28960 leader_election.cc:302] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ba7ec42e544896b1af7c407f6ebb27, 435632dda75647568ad27ca01dd6032f; no voters:
I20260808 03:02:21.718008 29386 raft_consensus.cc:2781] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:21.718498 29386 raft_consensus.cc:686] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 1 LEADER]: Becoming Leader. State: Replica: 41ba7ec42e544896b1af7c407f6ebb27, State: Running, Role: LEADER
I20260808 03:02:21.719017 29386 consensus_queue.cc:237] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 } } peers { permanent_uuid: "41ba7ec42e544896b1af7c407f6ebb27" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 38757 } } peers { permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 } }
I20260808 03:02:21.723575 28655 catalog_manager.cc:5495] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41ba7ec42e544896b1af7c407f6ebb27 (127.0.205.130). New cstate: current_term: 1 leader_uuid: "41ba7ec42e544896b1af7c407f6ebb27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ba7ec42e544896b1af7c407f6ebb27" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 38757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 } health_report { overall_health: UNKNOWN } } }
I20260808 03:02:21.766410 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.759987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45530 (local address 127.0.205.130:38757)
0808 03:02:21.760226 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:21.760234 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:21.760559 (+ 325us) server_negotiation.cc:407] Connection header received
0808 03:02:21.760808 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:21.760812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:21.760914 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:21.761077 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:02:21.761568 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.762363 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.763999 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.764450 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.764619 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.765203 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:02:21.765722 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:02:21.765957 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:21.766105 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 03:02:21.766167 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 03:02:21.774696 28871 raft_consensus.cc:1261] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c [term 1 FOLLOWER]: Refusing update from remote peer 41ba7ec42e544896b1af7c407f6ebb27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:21.774747 29276 raft_consensus.cc:1261] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f [term 1 FOLLOWER]: Refusing update from remote peer 41ba7ec42e544896b1af7c407f6ebb27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:21.775713 29386 consensus_queue.cc:1035] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1cc45b961748888d41094e9b0ad03c" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 35885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:21.776010 29393 consensus_queue.cc:1035] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435632dda75647568ad27ca01dd6032f" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 45383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:21.786455 29397 mvcc.cc:205] Tried to move back new op lower bound from 7316103748699480064 to 7316103748485234688. Current Snapshot: MvccSnapshot[applied={T|T < 7316103748699480064}]
I20260808 03:02:21.788679 29398 mvcc.cc:205] Tried to move back new op lower bound from 7316103748699480064 to 7316103748485234688. Current Snapshot: MvccSnapshot[applied={T|T < 7316103748699480064}]
I20260808 03:02:21.790782 29399 mvcc.cc:205] Tried to move back new op lower bound from 7316103748699480064 to 7316103748485234688. Current Snapshot: MvccSnapshot[applied={T|T < 7316103748699480064}]
I20260808 03:02:21.805672 28656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:21.806000 28656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:21.809651 28656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81eec70a991e43e588916e4322824a20: Sending DeleteTablet for 3 replicas of tablet 3b9914ebd0f946f4b85d5347912b36cc
I20260808 03:02:21.810936 29038 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9914ebd0f946f4b85d5347912b36cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:21 UTC) from {username='slave'} at 127.0.0.1:59056
I20260808 03:02:21.811146 28843 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9914ebd0f946f4b85d5347912b36cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:21 UTC) from {username='slave'} at 127.0.0.1:36278
I20260808 03:02:21.811751 29243 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9914ebd0f946f4b85d5347912b36cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:21 UTC) from {username='slave'} at 127.0.0.1:41020
I20260808 03:02:21.811923 29407 tablet_replica.cc:330] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: stopping tablet replica
I20260808 03:02:21.812500 29407 raft_consensus.cc:2229] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:02:21.813293 29407 raft_consensus.cc:2258] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:21.814193 29411 tablet_replica.cc:330] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: stopping tablet replica
I20260808 03:02:21.815101 29411 raft_consensus.cc:2229] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:21.815696 29411 raft_consensus.cc:2258] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:21.817288 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 28753
I20260808 03:02:21.818313 29411 ts_tablet_manager.cc:1905] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:21.826180 29411 ts_tablet_manager.cc:1918] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:21.826719 29407 ts_tablet_manager.cc:1905] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:21.827032 29410 tablet_replica.cc:330] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: stopping tablet replica
I20260808 03:02:21.827587 29410 raft_consensus.cc:2229] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:21.828261 29410 raft_consensus.cc:2258] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:21.830914 29410 ts_tablet_manager.cc:1905] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:21.835069 29407 ts_tablet_manager.cc:1918] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:21.835304 29407 log.cc:1196] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/3b9914ebd0f946f4b85d5347912b36cc
I20260808 03:02:21.835511 29411 log.cc:1196] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/3b9914ebd0f946f4b85d5347912b36cc
I20260808 03:02:21.835929 29407 ts_tablet_manager.cc:1939] T 3b9914ebd0f946f4b85d5347912b36cc P 41ba7ec42e544896b1af7c407f6ebb27: Deleting consensus metadata
I20260808 03:02:21.837519 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.818868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.0.205.129:35885)
0808 03:02:21.819107 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:21.819117 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:21.828599 (+ 9482us) server_negotiation.cc:407] Connection header received
0808 03:02:21.828859 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:21.828868 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:21.828989 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:21.829075 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:21.829825 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.830959 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.833003 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.833655 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.833767 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.835512 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:21.835524 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:21.835543 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:21.836426 (+ 883us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:21.836516 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:21.836542 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:21.836644 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:21.836938 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 03:02:21.837080 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 03:02:21.838070 28639 catalog_manager.cc:4841] TS 41ba7ec42e544896b1af7c407f6ebb27 (127.0.205.130:38757): tablet 3b9914ebd0f946f4b85d5347912b36cc (table test-table [id=6057857e575b4a5aa883413614412881]) successfully deleted
I20260808 03:02:21.838407 29411 ts_tablet_manager.cc:1939] T 3b9914ebd0f946f4b85d5347912b36cc P 435632dda75647568ad27ca01dd6032f: Deleting consensus metadata
I20260808 03:02:21.838912 28924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:21.840408 28638 catalog_manager.cc:4841] TS 435632dda75647568ad27ca01dd6032f (127.0.205.131:45383): tablet 3b9914ebd0f946f4b85d5347912b36cc (table test-table [id=6057857e575b4a5aa883413614412881]) successfully deleted
I20260808 03:02:21.932116 29410 ts_tablet_manager.cc:1918] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:21.932483 29410 log.cc:1196] T 3b9914ebd0f946f4b85d5347912b36cc P ce1cc45b961748888d41094e9b0ad03c: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/3b9914ebd0f946f4b85d5347912b36cc
I20260808 03:02:21.932988 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 28753
W20260808 03:02:21.960705 28640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.205.129:35885 (error 108)
I20260808 03:02:21.960973 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 28935
I20260808 03:02:21.963310 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.962216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:35885 (local address 127.0.0.1:54332)
0808 03:02:21.962760 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:21.962936 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:35885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":261,"threads_started":1}
W20260808 03:02:21.963711 28640 catalog_manager.cc:4584] TS ce1cc45b961748888d41094e9b0ad03c (127.0.205.129:35885): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b9914ebd0f946f4b85d5347912b36cc: Network error: Client connection negotiation failed: client connection to 127.0.205.129:35885: connect: Connection refused (error 111)
I20260808 03:02:21.970285 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:21.961920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45540 (local address 127.0.205.130:38757)
0808 03:02:21.962138 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:21.962146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:21.962649 (+ 503us) server_negotiation.cc:407] Connection header received
0808 03:02:21.962733 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:21.962741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:21.962868 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:21.963000 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:21.963554 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.964738 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.966495 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:21.967161 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:21.967302 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:21.968482 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:21.968494 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:21.968514 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:21.969359 (+ 845us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:21.969433 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:21.969460 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:21.969579 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:21.969756 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 03:02:21.969861 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 03:02:21.971127 29129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:22.017513 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:22.016916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:35885 (local address 127.0.0.1:54340)
0808 03:02:22.017253 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:22.017354 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:35885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260808 03:02:22.018705 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:22.018338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:35885 (local address 127.0.0.1:54344)
0808 03:02:22.018488 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:22.018559 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:35885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 03:02:22.079654 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:22.079106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:35885 (local address 127.0.0.1:54356)
0808 03:02:22.079356 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:22.079460 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:35885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260808 03:02:22.080804 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:22.080431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:35885 (local address 127.0.0.1:54360)
0808 03:02:22.080586 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:22.080660 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:35885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 03:02:22.087966 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 28935
I20260808 03:02:22.117324 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 29145
I20260808 03:02:22.124769 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:22.118214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.0.205.131:45383)
0808 03:02:22.118410 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:22.118419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:22.118443 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:02:22.118703 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:22.118708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:22.118797 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:22.118910 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:22.119345 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:22.120089 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:22.121410 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:22.121812 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:22.122165 (+ 353us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:22.123297 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:22.123306 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:22.123322 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:22.123915 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:22.123964 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:22.123984 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:22.124110 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:22.124254 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 03:02:22.124396 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:02:22.125460 29333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:22.161599 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:22.161103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:35885 (local address 127.0.0.1:54368)
0808 03:02:22.161337 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:22.161438 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:35885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 03:02:22.162774 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:22.162316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:35885 (local address 127.0.0.1:54378)
0808 03:02:22.162548 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:22.162631 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.129:35885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260808 03:02:22.218214 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 29145
I20260808 03:02:22.242755 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 28616
I20260808 03:02:22.243976 28706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:22.365593 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 28616
2026-08-08T03:02:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3370 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:22 dist-test-slave-6ntk krb5kdc[29424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:22 dist-test-slave-6ntk krb5kdc[29424](info): set up 2 sockets
Aug 08 03:02:22 dist-test-slave-6ntk krb5kdc[29424](info): commencing operation
krb5kdc: starting...
W20260808 03:02:23.785566 822 mini_kdc.cc:121] Time spent starting KDC: real 1.369s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:23 dist-test-slave-6ntk krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:02:24.121145 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:32857
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:32857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:24.486588 29445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:24.486936 29445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:24.487110 29445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:24.495227 29445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:24.495352 29445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:24.495438 29445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:24.495530 29445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:24.495572 29445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:24.506201 29445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:32857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:32857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:24.507625 29445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:24.508967 29445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:24.521464 29456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:24.521505 29462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:24.521436 29458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:24.522346 29445 server_base.cc:1034] running on GCE node
I20260808 03:02:24.523240 29445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:24.524749 29445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:24.526007 29445 hybrid_clock.cc:648] HybridClock initialized: now 1786158144525929 us; error 55 us; skew 500 ppm
I20260808 03:02:24.526386 29445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:24 dist-test-slave-6ntk krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158144, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:24.535172 29445 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:02:24.536793 29445 webserver.cc:460] Webserver started at http://127.0.205.190:34007/ using document root <none> and password file <none>
I20260808 03:02:24.537519 29445 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:24.537631 29445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:24.537999 29445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:24.541182 29445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "8a0df12261fd49489a3006ddc628180c"
format_stamp: "Formatted at 2026-08-08 03:02:24 on dist-test-slave-6ntk"
server_key: "077688416e0e33a43427fba070974e72"
server_key_iv: "1bde81875a418cb8bdae1bc30ee7ad7b"
server_key_version: "encryptionkey@0"
I20260808 03:02:24.542076 29445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "8a0df12261fd49489a3006ddc628180c"
format_stamp: "Formatted at 2026-08-08 03:02:24 on dist-test-slave-6ntk"
server_key: "077688416e0e33a43427fba070974e72"
server_key_iv: "1bde81875a418cb8bdae1bc30ee7ad7b"
server_key_version: "encryptionkey@0"
I20260808 03:02:24.547675 29445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 03:02:24.551433 29471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:24.552837 29445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:24.553005 29445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "8a0df12261fd49489a3006ddc628180c"
format_stamp: "Formatted at 2026-08-08 03:02:24 on dist-test-slave-6ntk"
server_key: "077688416e0e33a43427fba070974e72"
server_key_iv: "1bde81875a418cb8bdae1bc30ee7ad7b"
server_key_version: "encryptionkey@0"
I20260808 03:02:24.553188 29445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:24.582883 29445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:24.587646 29445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:24.587924 29445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:24.607820 29445 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:32857
I20260808 03:02:24.607843 29550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:32857 every 8 connection(s)
I20260808 03:02:24.609292 29445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:24.613274 29551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:24.614820 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 29445
I20260808 03:02:24.615036 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:24.615628 822 external_mini_cluster.cc:1428] Setting key 2d5ca26b4424198e1e0dd18a5abd6458
I20260808 03:02:24.629289 29551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c: Bootstrap starting.
Aug 08 03:02:24 dist-test-slave-6ntk krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:24.633198 29551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:24.634457 29551 log.cc:824] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:24.637634 29551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c: No bootstrap required, opened a new log
I20260808 03:02:24.640543 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:24.617752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58918 (local address 127.0.205.190:32857)
0808 03:02:24.618721 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:24.618735 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:24.618783 (+ 48us) server_negotiation.cc:407] Connection header received
0808 03:02:24.620490 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:24.620521 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:24.620972 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:24.621325 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:24.623319 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:24.624646 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:24.626695 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:24.627404 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:24.627548 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:24.632709 (+ 5161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:24.632718 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:24.632734 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:24.632780 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:24.636079 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:24.636885 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:24.636892 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:24.636899 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:24.637074 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:24.637531 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:24.637535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:24.637537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:24.638216 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:24.638563 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:24.639043 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 03:02:24.639502 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":738,"thread_start_us":438,"threads_started":1}
I20260808 03:02:24.646621 29551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0df12261fd49489a3006ddc628180c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 32857 } }
I20260808 03:02:24.647085 29551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:24.647219 29551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a0df12261fd49489a3006ddc628180c, State: Initialized, Role: FOLLOWER
I20260808 03:02:24.647806 29551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0df12261fd49489a3006ddc628180c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 32857 } }
I20260808 03:02:24.648083 29551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:24.648237 29551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:24.648430 29551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:24.651113 29551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0df12261fd49489a3006ddc628180c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 32857 } }
I20260808 03:02:24.651664 29551 leader_election.cc:302] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a0df12261fd49489a3006ddc628180c; no voters:
I20260808 03:02:24.652390 29551 leader_election.cc:288] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:24.652501 29557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:24.653286 29557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [term 1 LEADER]: Becoming Leader. State: Replica: 8a0df12261fd49489a3006ddc628180c, State: Running, Role: LEADER
I20260808 03:02:24.653973 29557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0df12261fd49489a3006ddc628180c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 32857 } }
I20260808 03:02:24.655018 29551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:24.657199 29559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a0df12261fd49489a3006ddc628180c. Latest consensus state: current_term: 1 leader_uuid: "8a0df12261fd49489a3006ddc628180c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0df12261fd49489a3006ddc628180c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 32857 } } }
I20260808 03:02:24.657603 29559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:24.658962 29558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a0df12261fd49489a3006ddc628180c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0df12261fd49489a3006ddc628180c" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 32857 } } }
I20260808 03:02:24.659541 29558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:24.663503 29565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:24.670248 29565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:24.679841 29565 catalog_manager.cc:1347] Generated new cluster ID: 6dfe518dffba4beaaf50fa0ddd83b0cc
I20260808 03:02:24.679991 29565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:24.698715 29565 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:24.700753 29565 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:24.717118 29565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c: Generated new TSK 0
I20260808 03:02:24.718348 29565 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.205.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
Entry for principal kudu/127.0.205.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab.
I20260808 03:02:24.798456 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:32857
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:25.156742 29586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:25.157092 29586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:25.157197 29586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:25.164974 29586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:25.165087 29586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:25.165205 29586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:02:25.175581 29586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.129.keytab
--principal=kudu/127.0.205.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:32857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:25.176910 29586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:25.178320 29586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:25.192677 29595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:25.192776 29598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:25.192739 29600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:25.193785 29586 server_base.cc:1034] running on GCE node
I20260808 03:02:25.194576 29586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:25.195703 29586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:25.196920 29586 hybrid_clock.cc:648] HybridClock initialized: now 1786158145196867 us; error 43 us; skew 500 ppm
I20260808 03:02:25.197276 29586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:25 dist-test-slave-6ntk krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:25.200858 29586 init.cc:377] Logged in from keytab as kudu/127.0.205.129@KRBTEST.COM (short username kudu)
I20260808 03:02:25.202414 29586 webserver.cc:460] Webserver started at http://127.0.205.129:43785/ using document root <none> and password file <none>
I20260808 03:02:25.203191 29586 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:25.203299 29586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:25.203588 29586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:25.206733 29586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374"
format_stamp: "Formatted at 2026-08-08 03:02:25 on dist-test-slave-6ntk"
server_key: "d43b5d984e5d7170001941374bb3714c"
server_key_iv: "17fcca51a8be9694f01ddccb889df8b4"
server_key_version: "encryptionkey@0"
I20260808 03:02:25.207520 29586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374"
format_stamp: "Formatted at 2026-08-08 03:02:25 on dist-test-slave-6ntk"
server_key: "d43b5d984e5d7170001941374bb3714c"
server_key_iv: "17fcca51a8be9694f01ddccb889df8b4"
server_key_version: "encryptionkey@0"
I20260808 03:02:25.213001 29586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 03:02:25.216675 29607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:25.218315 29586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:02:25.218509 29586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374"
format_stamp: "Formatted at 2026-08-08 03:02:25 on dist-test-slave-6ntk"
server_key: "d43b5d984e5d7170001941374bb3714c"
server_key_iv: "17fcca51a8be9694f01ddccb889df8b4"
server_key_version: "encryptionkey@0"
I20260808 03:02:25.218715 29586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:25.233654 29586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:25.237643 29586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:25.238010 29586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:25.239027 29586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:25.240404 29586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:25.240477 29586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:25.240605 29586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:25.240671 29586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:25.274828 29586 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:38351
I20260808 03:02:25.274881 29775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:38351 every 8 connection(s)
I20260808 03:02:25.276268 29586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:02:25.282146 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 29586
I20260808 03:02:25.282336 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 03:02:25.282929 822 external_mini_cluster.cc:1428] Setting key fe1177b264775b5a2a336b1d61995b66
Aug 08 03:02:25 dist-test-slave-6ntk krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.129@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:25.298127 29781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:25.279140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:41825 (local address 127.0.205.190:32857)
0808 03:02:25.279579 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:25.279584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:25.280596 (+ 1012us) server_negotiation.cc:407] Connection header received
0808 03:02:25.282625 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:25.282632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:25.282720 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:25.282852 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:25.284482 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:25.285228 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:25.286978 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:25.287399 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:25.287549 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:25.291743 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:25.291749 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:25.291752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:25.291789 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:25.294738 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:25.295582 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:25.295587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:25.295589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:25.295727 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:25.296226 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:25.296230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:25.296233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:25.296521 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:25.296688 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:25.297470 (+ 782us) server_negotiation.cc:299] Negotiation successful
0808 03:02:25.297718 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":213,"threads_started":1}
I20260808 03:02:25.299345 29782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:25.279711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:32857 (local address 127.0.205.129:41825)
0808 03:02:25.280418 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:25.280470 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:25.282255 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:25.283118 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:25.283127 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:25.283655 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:25.284276 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:25.284294 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:25.285405 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:25.285410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:25.286788 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:25.286801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:25.287647 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:25.287657 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:25.287916 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:25.288764 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:02:25.288790 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:25.291466 (+ 2676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:25.294926 (+ 3460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:25.294937 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:25.294961 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:25.295422 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:25.295856 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:25.295861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:25.295865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:25.296087 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:25.296660 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:25.296669 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:25.297055 (+ 386us) client_negotiation.cc:769] Sending connection context
0808 03:02:25.297337 (+ 282us) client_negotiation.cc:240] Negotiation successful
0808 03:02:25.297956 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":273,"threads_started":1}
I20260808 03:02:25.301115 29778 heartbeater.cc:347] Connected to a master server at 127.0.205.190:32857
I20260808 03:02:25.301478 29778 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:25.302529 29778 heartbeater.cc:511] Master 127.0.205.190:32857 requested a full tablet report, sending...
I20260808 03:02:25.305461 29501 ts_manager.cc:194] Registered new tserver with Master: 4ce7fdc0abaf4e4b89c41dfa5b36a374 (127.0.205.129:38351)
I20260808 03:02:25.307120 29501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.129@KRBTEST.COM'} at 127.0.205.129:41825
WARNING: no policy specified for kudu/127.0.205.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
Entry for principal kudu/127.0.205.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab.
I20260808 03:02:25.358109 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:32857
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:25.731464 29787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:25.731777 29787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:25.731873 29787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:25.739312 29787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:25.739442 29787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:25.739559 29787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:02:25.749929 29787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.130.keytab
--principal=kudu/127.0.205.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:32857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:25.751577 29787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:25.752868 29787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:25.766908 29795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:25.766995 29799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:25.766908 29797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:25.767663 29787 server_base.cc:1034] running on GCE node
I20260808 03:02:25.768410 29787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:25.769438 29787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:25.770653 29787 hybrid_clock.cc:648] HybridClock initialized: now 1786158145770591 us; error 47 us; skew 500 ppm
I20260808 03:02:25.771023 29787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:25 dist-test-slave-6ntk krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:25.774492 29787 init.cc:377] Logged in from keytab as kudu/127.0.205.130@KRBTEST.COM (short username kudu)
I20260808 03:02:25.776117 29787 webserver.cc:460] Webserver started at http://127.0.205.130:45333/ using document root <none> and password file <none>
I20260808 03:02:25.776875 29787 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:25.777032 29787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:25.777370 29787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:25.780445 29787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "69c998e8d79643c09a1cd5803caed70b"
format_stamp: "Formatted at 2026-08-08 03:02:25 on dist-test-slave-6ntk"
server_key: "4af1aa41700037877cacec913cf4bad7"
server_key_iv: "d828231d938f97b580f1f3675c6132c5"
server_key_version: "encryptionkey@0"
I20260808 03:02:25.781294 29787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "69c998e8d79643c09a1cd5803caed70b"
format_stamp: "Formatted at 2026-08-08 03:02:25 on dist-test-slave-6ntk"
server_key: "4af1aa41700037877cacec913cf4bad7"
server_key_iv: "d828231d938f97b580f1f3675c6132c5"
server_key_version: "encryptionkey@0"
I20260808 03:02:25.786638 29787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260808 03:02:25.790498 29808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:25.792052 29787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 03:02:25.792253 29787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "69c998e8d79643c09a1cd5803caed70b"
format_stamp: "Formatted at 2026-08-08 03:02:25 on dist-test-slave-6ntk"
server_key: "4af1aa41700037877cacec913cf4bad7"
server_key_iv: "d828231d938f97b580f1f3675c6132c5"
server_key_version: "encryptionkey@0"
I20260808 03:02:25.792479 29787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:25.828188 29787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:25.833046 29787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:25.833372 29787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:25.834434 29787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:25.835817 29787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:25.835942 29787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:25.836103 29787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:25.836179 29787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:25.871061 29787 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:42777
I20260808 03:02:25.871119 29985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:42777 every 8 connection(s)
I20260808 03:02:25.872463 29787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:02:25.872785 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 29787
I20260808 03:02:25.872989 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 03:02:25.873692 822 external_mini_cluster.cc:1428] Setting key 60db806b5a2a1dad5686c6bb16de90fd
Aug 08 03:02:25 dist-test-slave-6ntk krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.130@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:25.894196 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:25.875492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:55583 (local address 127.0.205.190:32857)
0808 03:02:25.875979 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:25.875987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:25.877067 (+ 1080us) server_negotiation.cc:407] Connection header received
0808 03:02:25.879060 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:25.879065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:25.879158 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:25.879278 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:25.880863 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:25.881574 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:25.883238 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:25.883679 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:25.883823 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:25.887506 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:25.887512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:25.887515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:25.887551 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:25.890385 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:25.891496 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:25.891501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:25.891503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:25.891651 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:25.892201 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:25.892204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:25.892206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:25.892490 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:25.892617 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:25.893651 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0808 03:02:25.893844 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":231,"threads_started":1}
I20260808 03:02:25.895869 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:25.876142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:32857 (local address 127.0.205.130:55583)
0808 03:02:25.876909 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:25.876955 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:25.878772 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:25.879543 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:25.879551 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:25.880094 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:25.880682 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:25.880701 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:25.881747 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:25.881751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:25.883086 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:25.883098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:25.883930 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:25.883937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:25.884175 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:25.884920 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:02:25.884943 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:25.887246 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:25.890558 (+ 3312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:25.890572 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:25.890594 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:25.891307 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:25.891778 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:25.891786 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:25.891793 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:25.892078 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:25.892626 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:25.892637 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:25.893156 (+ 519us) client_negotiation.cc:769] Sending connection context
0808 03:02:25.893560 (+ 404us) client_negotiation.cc:240] Negotiation successful
0808 03:02:25.894229 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":328,"threads_started":1}
I20260808 03:02:25.897554 29986 heartbeater.cc:347] Connected to a master server at 127.0.205.190:32857
I20260808 03:02:25.897917 29986 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:25.898700 29986 heartbeater.cc:511] Master 127.0.205.190:32857 requested a full tablet report, sending...
I20260808 03:02:25.900229 29501 ts_manager.cc:194] Registered new tserver with Master: 69c998e8d79643c09a1cd5803caed70b (127.0.205.130:42777)
I20260808 03:02:25.901091 29501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.130@KRBTEST.COM'} at 127.0.205.130:55583
WARNING: no policy specified for kudu/127.0.205.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
Entry for principal kudu/127.0.205.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab.
I20260808 03:02:25.949266 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:32857
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:02:26.311008 29778 heartbeater.cc:503] Master 127.0.205.190:32857 was elected leader, sending a full tablet report...
W20260808 03:02:26.315728 29993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:26.316062 29993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:26.316264 29993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:26.323798 29993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:26.323961 29993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:26.324133 29993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:02:26.334164 29993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/krb5kdc/kudu_127.0.205.131.keytab
--principal=kudu/127.0.205.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:32857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:26.335573 29993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:26.336961 29993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:26.351009 30001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:26.350929 30002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:26.350975 30005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:26.352047 29993 server_base.cc:1034] running on GCE node
I20260808 03:02:26.352653 29993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:26.353649 29993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:26.354854 29993 hybrid_clock.cc:648] HybridClock initialized: now 1786158146354799 us; error 54 us; skew 500 ppm
I20260808 03:02:26.355235 29993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:26 dist-test-slave-6ntk krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:26.358817 29993 init.cc:377] Logged in from keytab as kudu/127.0.205.131@KRBTEST.COM (short username kudu)
I20260808 03:02:26.360548 29993 webserver.cc:460] Webserver started at http://127.0.205.131:36503/ using document root <none> and password file <none>
I20260808 03:02:26.361270 29993 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:26.361423 29993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:26.361773 29993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:26.364934 29993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "ddd858503b01419b8e205e6c36416642"
format_stamp: "Formatted at 2026-08-08 03:02:26 on dist-test-slave-6ntk"
server_key: "ddf7bfe4f0b3bd82c35b16a5d82cc57c"
server_key_iv: "a5b8c8fb7c70bbc60c513be1269301b6"
server_key_version: "encryptionkey@0"
I20260808 03:02:26.365759 29993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "ddd858503b01419b8e205e6c36416642"
format_stamp: "Formatted at 2026-08-08 03:02:26 on dist-test-slave-6ntk"
server_key: "ddf7bfe4f0b3bd82c35b16a5d82cc57c"
server_key_iv: "a5b8c8fb7c70bbc60c513be1269301b6"
server_key_version: "encryptionkey@0"
I20260808 03:02:26.371125 29993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 03:02:26.374814 30017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:26.376305 29993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:26.376514 29993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "ddd858503b01419b8e205e6c36416642"
format_stamp: "Formatted at 2026-08-08 03:02:26 on dist-test-slave-6ntk"
server_key: "ddf7bfe4f0b3bd82c35b16a5d82cc57c"
server_key_iv: "a5b8c8fb7c70bbc60c513be1269301b6"
server_key_version: "encryptionkey@0"
I20260808 03:02:26.376724 29993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:26.397547 29993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:26.401856 29993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:26.402261 29993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:26.403280 29993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:26.404836 29993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:26.404945 29993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:26.405100 29993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:26.405174 29993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:26.440485 29993 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:39883
I20260808 03:02:26.440532 30195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:39883 every 8 connection(s)
I20260808 03:02:26.441857 29993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:02:26.443300 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 29993
I20260808 03:02:26.443536 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 03:02:26.444124 822 external_mini_cluster.cc:1428] Setting key f7dd95ceda9997a8e9713c8ff206ef56
Aug 08 03:02:26 dist-test-slave-6ntk krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.131@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:02:26.463675 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.444959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:59405 (local address 127.0.205.190:32857)
0808 03:02:26.445455 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:26.445463 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:26.446553 (+ 1090us) server_negotiation.cc:407] Connection header received
0808 03:02:26.448708 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:26.448713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:26.448814 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:26.448947 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:26.450550 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.451334 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.452906 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.453430 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.453567 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.457481 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:26.457487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:26.457490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:26.457523 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:26.460457 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:26.461213 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:26.461226 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:26.461229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:26.461354 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:26.461816 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:26.461825 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:26.461827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:26.462159 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:26.462333 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:26.463040 (+ 707us) server_negotiation.cc:299] Negotiation successful
0808 03:02:26.463262 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":254,"threads_started":1}
I20260808 03:02:26.464725 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.445392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:32857 (local address 127.0.205.131:59405)
0808 03:02:26.446370 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:26.446432 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:26.448444 (+ 2012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:26.449196 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:26.449205 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:26.449728 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:26.450380 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.450398 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.451507 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.451511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.452750 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.452762 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.453660 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.453668 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.453964 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.454710 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:02:26.454739 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:26.457250 (+ 2511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:26.460611 (+ 3361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:26.460620 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:26.460638 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:26.461078 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:26.461474 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:02:26.461479 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:02:26.461483 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:02:26.461690 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:02:26.462293 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:26.462302 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:26.462659 (+ 357us) client_negotiation.cc:769] Sending connection context
0808 03:02:26.462941 (+ 282us) client_negotiation.cc:240] Negotiation successful
0808 03:02:26.463492 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":434,"threads_started":1}
I20260808 03:02:26.466156 30196 heartbeater.cc:347] Connected to a master server at 127.0.205.190:32857
I20260808 03:02:26.466506 30196 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:26.467259 30196 heartbeater.cc:511] Master 127.0.205.190:32857 requested a full tablet report, sending...
I20260808 03:02:26.468803 29501 ts_manager.cc:194] Registered new tserver with Master: ddd858503b01419b8e205e6c36416642 (127.0.205.131:39883)
I20260808 03:02:26.469719 29501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.205.131@KRBTEST.COM'} at 127.0.205.131:59405
I20260808 03:02:26.472476 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:02:26.489521 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.476699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58932 (local address 127.0.205.190:32857)
0808 03:02:26.476951 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:26.476960 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:26.477266 (+ 306us) server_negotiation.cc:407] Connection header received
0808 03:02:26.477391 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:26.477394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:26.477471 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:26.477561 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:26.479053 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.479761 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.481164 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.481579 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.481721 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.484120 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:26.484125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:26.484128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:26.484162 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:26.486987 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:26.487663 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:26.487667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:26.487669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:26.487810 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:26.488141 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:26.488144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:26.488146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:26.488508 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:26.488696 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:26.489001 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 03:02:26.489173 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 03:02:26.493950 29501 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:02:26.501575 29501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:02:26.540522 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.514775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32910 (local address 127.0.205.131:39883)
0808 03:02:26.531234 (+ 16459us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:26.531243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:26.531281 (+ 38us) server_negotiation.cc:407] Connection header received
0808 03:02:26.531391 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:26.531399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:26.531687 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:26.531933 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:26.533049 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.534681 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.537259 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.538783 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.538931 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.539202 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:26.540008 (+ 806us) server_negotiation.cc:299] Negotiation successful
0808 03:02:26.540179 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16283,"thread_start_us":16121,"threads_started":1}
I20260808 03:02:26.540589 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.514578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:39883 (local address 127.0.0.1:32910)
0808 03:02:26.526829 (+ 12251us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:26.526898 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:26.527185 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:26.532283 (+ 5098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:26.532291 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:26.532320 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:26.532812 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.532831 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.534885 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.534890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.537074 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.537087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.539342 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.539352 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.539657 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.539784 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 03:02:26.539933 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 03:02:26.540154 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11832,"mutex_wait_us":114,"thread_start_us":16505,"threads_started":1}
I20260808 03:02:26.540596 30221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.528052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53418 (local address 127.0.205.129:38351)
0808 03:02:26.530433 (+ 2381us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:26.530444 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:26.531734 (+ 1290us) server_negotiation.cc:407] Connection header received
0808 03:02:26.531958 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:26.531967 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:26.532202 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:26.532440 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:26.533233 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.534523 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.537514 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.538956 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.539081 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.539289 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:26.540089 (+ 800us) server_negotiation.cc:299] Negotiation successful
0808 03:02:26.540254 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2189,"thread_start_us":2019,"threads_started":1}
I20260808 03:02:26.540882 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.514953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.0.205.130:42777)
0808 03:02:26.530486 (+ 15533us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:26.530496 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:26.532049 (+ 1553us) server_negotiation.cc:407] Connection header received
0808 03:02:26.532164 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:26.532171 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:26.532437 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:26.532637 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:26.533621 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.535227 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.537950 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.539318 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.539461 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.539693 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:26.540341 (+ 648us) server_negotiation.cc:299] Negotiation successful
0808 03:02:26.540522 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15323,"thread_start_us":15547,"threads_started":1}
I20260808 03:02:26.541370 30217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.514654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:42777 (local address 127.0.0.1:43508)
0808 03:02:26.530805 (+ 16151us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:26.530828 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:26.530990 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:26.532890 (+ 1900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:26.532896 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:26.532928 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:26.533423 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.533436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.535542 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.535551 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.537765 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.537775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.539595 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.539601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.539865 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.539890 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 03:02:26.540006 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 03:02:26.540153 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16001,"mutex_wait_us":99,"thread_start_us":11558,"threads_started":1}
I20260808 03:02:26.541402 30222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.530574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:38351 (local address 127.0.0.1:53418)
0808 03:02:26.531598 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:26.531612 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:26.531773 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:26.532425 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:26.532430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:26.532462 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:26.532923 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.532960 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.534691 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.534699 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.537334 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.537351 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.539456 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.539468 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.539652 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.539783 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 03:02:26.539926 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 03:02:26.540951 (+ 1025us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"mutex_wait_us":865,"thread_start_us":680,"threads_started":1}
I20260808 03:02:26.545295 29891 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca4c30f243f42dea660d34be72e7190 (DEFAULT_TABLE table=test-table [id=c8fc8fce85de4b56a0908f2bb36bb773]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:26.545317 30090 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca4c30f243f42dea660d34be72e7190 (DEFAULT_TABLE table=test-table [id=c8fc8fce85de4b56a0908f2bb36bb773]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:26.545312 29682 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca4c30f243f42dea660d34be72e7190 (DEFAULT_TABLE table=test-table [id=c8fc8fce85de4b56a0908f2bb36bb773]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:26.546677 29891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca4c30f243f42dea660d34be72e7190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:26.546821 29682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca4c30f243f42dea660d34be72e7190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:26.546821 30090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca4c30f243f42dea660d34be72e7190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:26.559082 30224 tablet_bootstrap.cc:492] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: Bootstrap starting.
I20260808 03:02:26.562042 30227 tablet_bootstrap.cc:492] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: Bootstrap starting.
I20260808 03:02:26.562884 30226 tablet_bootstrap.cc:492] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: Bootstrap starting.
I20260808 03:02:26.565109 30224 tablet_bootstrap.cc:654] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:26.566529 30224 log.cc:824] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:26.566890 30227 tablet_bootstrap.cc:654] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:26.568018 30227 log.cc:824] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:26.569255 30226 tablet_bootstrap.cc:654] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:26.570029 30224 tablet_bootstrap.cc:492] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: No bootstrap required, opened a new log
I20260808 03:02:26.570328 30224 ts_tablet_manager.cc:1397] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260808 03:02:26.570384 30226 log.cc:824] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:26.571281 30227 tablet_bootstrap.cc:492] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: No bootstrap required, opened a new log
I20260808 03:02:26.571650 30227 ts_tablet_manager.cc:1397] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 03:02:26.575181 30226 tablet_bootstrap.cc:492] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: No bootstrap required, opened a new log
I20260808 03:02:26.575496 30226 ts_tablet_manager.cc:1397] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260808 03:02:26.581849 30224 raft_consensus.cc:348] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 } } peers { permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 } } peers { permanent_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38351 } }
I20260808 03:02:26.582587 30224 raft_consensus.cc:374] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:26.582762 30224 raft_consensus.cc:729] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c998e8d79643c09a1cd5803caed70b, State: Initialized, Role: FOLLOWER
I20260808 03:02:26.582700 30227 raft_consensus.cc:348] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 } } peers { permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 } } peers { permanent_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38351 } }
I20260808 03:02:26.583237 30227 raft_consensus.cc:374] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:26.583388 30227 raft_consensus.cc:729] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ce7fdc0abaf4e4b89c41dfa5b36a374, State: Initialized, Role: FOLLOWER
I20260808 03:02:26.583639 30224 consensus_queue.cc:260] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 } } peers { permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 } } peers { permanent_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38351 } }
I20260808 03:02:26.583740 30226 raft_consensus.cc:348] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 } } peers { permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 } } peers { permanent_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38351 } }
I20260808 03:02:26.584215 30226 raft_consensus.cc:374] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:26.584065 30227 consensus_queue.cc:260] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 } } peers { permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 } } peers { permanent_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38351 } }
I20260808 03:02:26.584339 30226 raft_consensus.cc:729] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd858503b01419b8e205e6c36416642, State: Initialized, Role: FOLLOWER
I20260808 03:02:26.584887 30226 consensus_queue.cc:260] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 } } peers { permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 } } peers { permanent_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38351 } }
I20260808 03:02:26.586216 30196 heartbeater.cc:503] Master 127.0.205.190:32857 was elected leader, sending a full tablet report...
I20260808 03:02:26.586751 30226 ts_tablet_manager.cc:1428] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260808 03:02:26.588640 30227 ts_tablet_manager.cc:1428] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
I20260808 03:02:26.592711 29986 heartbeater.cc:503] Master 127.0.205.190:32857 was elected leader, sending a full tablet report...
I20260808 03:02:26.594501 30224 ts_tablet_manager.cc:1428] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: Time spent starting tablet: real 0.024s user 0.012s sys 0.010s
W20260808 03:02:26.624826 29988 tablet.cc:2369] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:02:26.693708 30198 tablet.cc:2369] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:02:26.720476 30232 raft_consensus.cc:482] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:02:26.720830 30232 raft_consensus.cc:504] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 } } peers { permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 } } peers { permanent_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38351 } }
I20260808 03:02:26.722764 30232 leader_election.cc:288] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddd858503b01419b8e205e6c36416642 (127.0.205.131:39883), 69c998e8d79643c09a1cd5803caed70b (127.0.205.130:42777)
I20260808 03:02:26.730222 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.723141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:34645 (local address 127.0.205.131:39883)
0808 03:02:26.723386 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:26.723395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:26.724576 (+ 1181us) server_negotiation.cc:407] Connection header received
0808 03:02:26.724781 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:26.724785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:26.724843 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:26.724951 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:26.725688 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.726507 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.728448 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.729226 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.729403 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.729486 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:26.729786 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 03:02:26.729955 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 03:02:26.730288 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.723489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:54963 (local address 127.0.205.130:42777)
0808 03:02:26.723982 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:26.723990 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:26.724340 (+ 350us) server_negotiation.cc:407] Connection header received
0808 03:02:26.724668 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:26.724674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:26.724750 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:26.724813 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:26.725778 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.726543 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.728472 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.729300 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.729491 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.729575 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:26.729858 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 03:02:26.730019 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318}
I20260808 03:02:26.730405 30239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.723149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:39883 (local address 127.0.205.129:34645)
0808 03:02:26.724476 (+ 1327us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:26.724495 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:26.724629 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:26.724971 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:26.724975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:26.725000 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:26.725525 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.725537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.726634 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.726641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.728282 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.728294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.729464 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.729470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.729632 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.729655 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 03:02:26.729761 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 03:02:26.729954 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1201,"mutex_wait_us":122,"thread_start_us":614,"threads_started":1}
I20260808 03:02:26.730465 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.723047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:42777 (local address 127.0.205.129:54963)
0808 03:02:26.724224 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:26.724245 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:26.724467 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:26.725046 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:26.725049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:26.725120 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:26.725616 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.725628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.726691 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.726698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.728272 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:26.728284 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.729546 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.729552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:26.729698 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.729720 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 03:02:26.729827 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 03:02:26.730017 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"mutex_wait_us":54,"spinlock_wait_cycles":20352,"thread_start_us":1112,"threads_started":1}
I20260808 03:02:26.731532 30119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca4c30f243f42dea660d34be72e7190" candidate_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd858503b01419b8e205e6c36416642" is_pre_election: true
I20260808 03:02:26.731534 29915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca4c30f243f42dea660d34be72e7190" candidate_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c998e8d79643c09a1cd5803caed70b" is_pre_election: true
I20260808 03:02:26.732050 29915 raft_consensus.cc:2454] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ce7fdc0abaf4e4b89c41dfa5b36a374 in term 0.
I20260808 03:02:26.732050 30119 raft_consensus.cc:2454] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ce7fdc0abaf4e4b89c41dfa5b36a374 in term 0.
I20260808 03:02:26.732740 29613 leader_election.cc:302] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ce7fdc0abaf4e4b89c41dfa5b36a374, 69c998e8d79643c09a1cd5803caed70b; no voters:
I20260808 03:02:26.733150 30232 raft_consensus.cc:2781] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:02:26.733304 30232 raft_consensus.cc:482] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:02:26.733422 30232 raft_consensus.cc:3037] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:26.735678 30232 raft_consensus.cc:504] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 } } peers { permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 } } peers { permanent_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38351 } }
I20260808 03:02:26.736259 30232 leader_election.cc:288] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [CANDIDATE]: Term 1 election: Requested vote from peers ddd858503b01419b8e205e6c36416642 (127.0.205.131:39883), 69c998e8d79643c09a1cd5803caed70b (127.0.205.130:42777)
I20260808 03:02:26.736711 30119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca4c30f243f42dea660d34be72e7190" candidate_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd858503b01419b8e205e6c36416642"
I20260808 03:02:26.736954 30119 raft_consensus.cc:3037] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:26.736979 29915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca4c30f243f42dea660d34be72e7190" candidate_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69c998e8d79643c09a1cd5803caed70b"
I20260808 03:02:26.737181 29915 raft_consensus.cc:3037] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:26.739413 30119 raft_consensus.cc:2454] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ce7fdc0abaf4e4b89c41dfa5b36a374 in term 1.
I20260808 03:02:26.739534 29915 raft_consensus.cc:2454] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ce7fdc0abaf4e4b89c41dfa5b36a374 in term 1.
I20260808 03:02:26.740033 29608 leader_election.cc:302] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ce7fdc0abaf4e4b89c41dfa5b36a374, ddd858503b01419b8e205e6c36416642; no voters:
I20260808 03:02:26.740396 30232 raft_consensus.cc:2781] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:26.740875 30232 raft_consensus.cc:686] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 1 LEADER]: Becoming Leader. State: Replica: 4ce7fdc0abaf4e4b89c41dfa5b36a374, State: Running, Role: LEADER
I20260808 03:02:26.741438 30232 consensus_queue.cc:237] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 } } peers { permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 } } peers { permanent_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38351 } }
I20260808 03:02:26.746501 29501 catalog_manager.cc:5495] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ce7fdc0abaf4e4b89c41dfa5b36a374 (127.0.205.129). New cstate: current_term: 1 leader_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ce7fdc0abaf4e4b89c41dfa5b36a374" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 38351 } health_report { overall_health: HEALTHY } } }
W20260808 03:02:26.779009 29780 tablet.cc:2369] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:02:26.784242 30221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.778365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53430 (local address 127.0.205.129:38351)
0808 03:02:26.778615 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:26.778619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:26.778635 (+ 16us) server_negotiation.cc:407] Connection header received
0808 03:02:26.778771 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:26.778777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:26.778914 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:26.779096 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:02:26.779561 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.780547 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.782154 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.782556 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.782655 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.783113 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:02:26.783659 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:02:26.783772 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:26.783863 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 03:02:26.783936 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:02:26.793143 29915 raft_consensus.cc:1261] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b [term 1 FOLLOWER]: Refusing update from remote peer 4ce7fdc0abaf4e4b89c41dfa5b36a374: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:26.793143 30119 raft_consensus.cc:1261] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642 [term 1 FOLLOWER]: Refusing update from remote peer 4ce7fdc0abaf4e4b89c41dfa5b36a374: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:26.794289 30241 consensus_queue.cc:1035] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69c998e8d79643c09a1cd5803caed70b" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 42777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:26.794680 30232 consensus_queue.cc:1035] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd858503b01419b8e205e6c36416642" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 39883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:26.811547 30247 mvcc.cc:205] Tried to move back new op lower bound from 7316103769254064128 to 7316103769057394688. Current Snapshot: MvccSnapshot[applied={T|T < 7316103769254064128}]
I20260808 03:02:26.824909 29499 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:26.825246 29499 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:26.829370 29499 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a0df12261fd49489a3006ddc628180c: Sending DeleteTablet for 3 replicas of tablet 9ca4c30f243f42dea660d34be72e7190
I20260808 03:02:26.830816 30090 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca4c30f243f42dea660d34be72e7190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:26 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:32910
I20260808 03:02:26.830885 29891 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca4c30f243f42dea660d34be72e7190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:26 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:43508
I20260808 03:02:26.831851 30256 tablet_replica.cc:330] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: stopping tablet replica
I20260808 03:02:26.832341 30256 raft_consensus.cc:2229] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:26.832919 30256 raft_consensus.cc:2258] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:26.832827 29682 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca4c30f243f42dea660d34be72e7190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:26 UTC) from {username='kudu', principal='kudu/127.0.205.190@KRBTEST.COM'} at 127.0.0.1:53418
I20260808 03:02:26.833752 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 29586
I20260808 03:02:26.835418 30256 ts_tablet_manager.cc:1905] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:26.844480 30256 ts_tablet_manager.cc:1918] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:26.844755 30256 log.cc:1196] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/9ca4c30f243f42dea660d34be72e7190
I20260808 03:02:26.845439 30256 ts_tablet_manager.cc:1939] T 9ca4c30f243f42dea660d34be72e7190 P 69c998e8d79643c09a1cd5803caed70b: Deleting consensus metadata
I20260808 03:02:26.847879 29479 catalog_manager.cc:4841] TS 69c998e8d79643c09a1cd5803caed70b (127.0.205.130:42777): tablet 9ca4c30f243f42dea660d34be72e7190 (table test-table [id=c8fc8fce85de4b56a0908f2bb36bb773]) successfully deleted
I20260808 03:02:26.848891 30258 tablet_replica.cc:330] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: stopping tablet replica
I20260808 03:02:26.850237 30258 raft_consensus.cc:2229] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:02:26.850508 30257 tablet_replica.cc:330] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: stopping tablet replica
I20260808 03:02:26.850992 30257 raft_consensus.cc:2229] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:26.851091 30258 raft_consensus.cc:2258] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:26.851665 30257 raft_consensus.cc:2258] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:26.853605 30258 ts_tablet_manager.cc:1905] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:26.853602 30257 ts_tablet_manager.cc:1905] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 03:02:26 dist-test-slave-6ntk krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.129@KRBTEST.COM
I20260808 03:02:26.862337 30257 ts_tablet_manager.cc:1918] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:26.862654 30257 log.cc:1196] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/9ca4c30f243f42dea660d34be72e7190
I20260808 03:02:26.863426 30257 ts_tablet_manager.cc:1939] T 9ca4c30f243f42dea660d34be72e7190 P ddd858503b01419b8e205e6c36416642: Deleting consensus metadata
I20260808 03:02:26.863322 30258 ts_tablet_manager.cc:1918] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:26.864008 30258 log.cc:1196] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/9ca4c30f243f42dea660d34be72e7190
I20260808 03:02:26.865124 30258 ts_tablet_manager.cc:1939] T 9ca4c30f243f42dea660d34be72e7190 P 4ce7fdc0abaf4e4b89c41dfa5b36a374: Deleting consensus metadata
I20260808 03:02:26.866017 29473 catalog_manager.cc:4841] TS ddd858503b01419b8e205e6c36416642 (127.0.205.131:39883): tablet 9ca4c30f243f42dea660d34be72e7190 (table test-table [id=c8fc8fce85de4b56a0908f2bb36bb773]) successfully deleted
I20260808 03:02:26.867295 29479 catalog_manager.cc:4841] TS 4ce7fdc0abaf4e4b89c41dfa5b36a374 (127.0.205.129:38351): tablet 9ca4c30f243f42dea660d34be72e7190 (table test-table [id=c8fc8fce85de4b56a0908f2bb36bb773]) successfully deleted
I20260808 03:02:26.870726 30221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:26.836458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53446 (local address 127.0.205.129:38351)
0808 03:02:26.836657 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:26.836663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:26.849531 (+ 12868us) server_negotiation.cc:407] Connection header received
0808 03:02:26.849751 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:26.849757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:26.849926 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:26.850038 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:26.851911 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.853112 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.855447 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:26.856270 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:26.856386 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:26.862477 (+ 6091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:26.862499 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:26.862519 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:26.862579 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:26.867248 (+ 4669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:26.868274 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:26.868282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:26.868284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:26.868454 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:26.868914 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:26.868918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:26.868920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:26.869295 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:26.869539 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:26.869998 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 03:02:26.870227 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 03:02:26.874670 29770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:26.977648 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 29586
I20260808 03:02:27.008453 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 29787
Aug 08 03:02:27 dist-test-slave-6ntk krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.130@KRBTEST.COM
I20260808 03:02:27.024478 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:27.009322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.0.205.130:42777)
0808 03:02:27.009532 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:27.009540 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:27.009563 (+ 23us) server_negotiation.cc:407] Connection header received
0808 03:02:27.009780 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:27.009785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:27.009862 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:27.010006 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:27.011333 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:27.012105 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:27.013436 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:27.013851 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:27.014145 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:27.018554 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:27.018569 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:27.018585 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:27.018627 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:27.021412 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:27.022203 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:27.022220 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:27.022223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:27.022357 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:27.022802 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:27.022809 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:27.022812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:27.023252 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:27.023575 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:27.023885 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 03:02:27.024115 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 03:02:27.025224 29977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:27.118280 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 29787
I20260808 03:02:27.144357 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 29993
Aug 08 03:02:27 dist-test-slave-6ntk krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.131@KRBTEST.COM
I20260808 03:02:27.160475 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:27.145196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32914 (local address 127.0.205.131:39883)
0808 03:02:27.145398 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:27.145403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:27.145423 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:02:27.145500 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:27.145504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:27.145565 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:27.145643 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:27.147215 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:27.147945 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:27.149241 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:27.149622 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:27.149718 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:27.154435 (+ 4717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:27.154454 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:27.154472 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:27.154538 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:27.157674 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:27.158526 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:27.158534 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:27.158536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:27.158683 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:27.159046 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:27.159050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:27.159052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:27.159427 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:27.159649 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:27.159908 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 03:02:27.160083 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 03:02:27.161194 30184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:27.253963 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 29993
I20260808 03:02:27.280067 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 29445
I20260808 03:02:27.281283 29544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:27.400249 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 29445
2026-08-08T03:02:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5035 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T03:02:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:27Z Disabled control of system clock
I20260808 03:02:27.502820 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:40233
--webserver_interface=127.0.205.190
--webserver_port=0
--builtin_ntp_servers=127.0.205.148:33039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:40233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:02:27.870087 30274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:27.870400 30274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:27.870484 30274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:27.877691 30274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:27.877801 30274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:27.877840 30274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:27.877961 30274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:27.878018 30274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:27.887413 30274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:33039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:40233
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:40233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:27.888736 30274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:27.890555 30274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:27.902979 30286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:27.903239 30284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:27.903456 30282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:27.905138 30274 server_base.cc:1034] running on GCE node
I20260808 03:02:27.906188 30274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:27.907764 30274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:27.908983 30274 hybrid_clock.cc:648] HybridClock initialized: now 1786158147908937 us; error 44 us; skew 500 ppm
I20260808 03:02:27.909366 30274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:27.911950 30274 webserver.cc:460] Webserver started at http://127.0.205.190:41191/ using document root <none> and password file <none>
I20260808 03:02:27.912704 30274 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:27.912824 30274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:27.913187 30274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:27.916337 30274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "36819fcb1efd47a39209e8928dd533f6"
format_stamp: "Formatted at 2026-08-08 03:02:27 on dist-test-slave-6ntk"
server_key: "3db09e40320fd44e5117b417dd861dc3"
server_key_iv: "b37c32044a15975779f0ae814c6b5bcd"
server_key_version: "encryptionkey@0"
I20260808 03:02:27.917189 30274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "36819fcb1efd47a39209e8928dd533f6"
format_stamp: "Formatted at 2026-08-08 03:02:27 on dist-test-slave-6ntk"
server_key: "3db09e40320fd44e5117b417dd861dc3"
server_key_iv: "b37c32044a15975779f0ae814c6b5bcd"
server_key_version: "encryptionkey@0"
I20260808 03:02:27.922730 30274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 03:02:27.926306 30294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:27.927735 30274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 03:02:27.927944 30274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "36819fcb1efd47a39209e8928dd533f6"
format_stamp: "Formatted at 2026-08-08 03:02:27 on dist-test-slave-6ntk"
server_key: "3db09e40320fd44e5117b417dd861dc3"
server_key_iv: "b37c32044a15975779f0ae814c6b5bcd"
server_key_version: "encryptionkey@0"
I20260808 03:02:27.928216 30274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:27.952379 30274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:27.953334 30274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:27.953658 30274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:27.973712 30274 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:40233
I20260808 03:02:27.973735 30377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:40233 every 8 connection(s)
I20260808 03:02:27.975220 30274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:27.976053 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 30274
I20260808 03:02:27.976271 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:27.976900 822 external_mini_cluster.cc:1428] Setting key 179ab46a1825fe647b3d9e3df7ac37e9
I20260808 03:02:27.980027 30379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:27.991600 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:27.978995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.0.205.190:40233)
0808 03:02:27.980037 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:27.980048 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:27.980092 (+ 44us) server_negotiation.cc:407] Connection header received
0808 03:02:27.981830 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:27.981851 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:27.982350 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:27.982708 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:27.983346 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:27.984755 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:27.986834 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:27.987549 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:27.987684 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:27.989113 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:27.989120 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:27.989139 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:27.989758 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:27.989838 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:27.989914 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:27.990036 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:27.990267 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 03:02:27.990628 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":403,"threads_started":1}
I20260808 03:02:27.992762 30379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6: Bootstrap starting.
I20260808 03:02:27.996217 30379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:27.997254 30379 log.cc:824] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:28.000432 30379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6: No bootstrap required, opened a new log
I20260808 03:02:28.008224 30379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36819fcb1efd47a39209e8928dd533f6" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40233 } }
I20260808 03:02:28.008719 30379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:28.008826 30379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36819fcb1efd47a39209e8928dd533f6, State: Initialized, Role: FOLLOWER
I20260808 03:02:28.009371 30379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36819fcb1efd47a39209e8928dd533f6" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40233 } }
I20260808 03:02:28.009589 30379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:28.009696 30379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:28.009837 30379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:28.012167 30379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36819fcb1efd47a39209e8928dd533f6" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40233 } }
I20260808 03:02:28.012662 30379 leader_election.cc:302] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36819fcb1efd47a39209e8928dd533f6; no voters:
I20260808 03:02:28.013262 30379 leader_election.cc:288] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:28.013463 30387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:28.014762 30387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [term 1 LEADER]: Becoming Leader. State: Replica: 36819fcb1efd47a39209e8928dd533f6, State: Running, Role: LEADER
I20260808 03:02:28.015437 30387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36819fcb1efd47a39209e8928dd533f6" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40233 } }
I20260808 03:02:28.016265 30379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:28.018684 30389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36819fcb1efd47a39209e8928dd533f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36819fcb1efd47a39209e8928dd533f6" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40233 } } }
I20260808 03:02:28.019122 30389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:28.023439 30390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36819fcb1efd47a39209e8928dd533f6. Latest consensus state: current_term: 1 leader_uuid: "36819fcb1efd47a39209e8928dd533f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36819fcb1efd47a39209e8928dd533f6" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40233 } } }
I20260808 03:02:28.024036 30390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:28.024631 30397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:28.029012 30397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:28.037652 30397 catalog_manager.cc:1347] Generated new cluster ID: 1209b4e2477240ec939d538e9262fab1
I20260808 03:02:28.037784 30397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:28.046608 30397 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:28.048519 30397 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:28.063513 30397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6: Generated new TSK 0
I20260808 03:02:28.064518 30397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:28.079722 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.129:0
--local_ip_for_outbound_sockets=127.0.205.129
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:40233
--builtin_ntp_servers=127.0.205.148:33039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 03:02:28.449378 30419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:28.449687 30419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:28.449777 30419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:28.457240 30419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:28.457397 30419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:28.457572 30419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.129
I20260808 03:02:28.468344 30419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:33039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.205.129
--webserver_port=0
--tserver_master_addrs=127.0.205.190:40233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.129
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:28.469918 30419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:28.471916 30419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:28.486636 30428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:28.486760 30425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:28.486720 30426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:28.487876 30419 server_base.cc:1034] running on GCE node
I20260808 03:02:28.488538 30419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:28.489544 30419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:28.490767 30419 hybrid_clock.cc:648] HybridClock initialized: now 1786158148490711 us; error 54 us; skew 500 ppm
I20260808 03:02:28.491088 30419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:28.493474 30419 webserver.cc:460] Webserver started at http://127.0.205.129:33709/ using document root <none> and password file <none>
I20260808 03:02:28.494298 30419 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:28.494407 30419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:28.494737 30419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:28.497824 30419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/data/instance:
uuid: "f059ce1b55fe421594cd51754ae4215e"
format_stamp: "Formatted at 2026-08-08 03:02:28 on dist-test-slave-6ntk"
server_key: "fbbab5d512278fea015163b307f51bd3"
server_key_iv: "a8bebb7de8ac71f27a21e33e1ba90124"
server_key_version: "encryptionkey@0"
I20260808 03:02:28.498854 30419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/wal/instance:
uuid: "f059ce1b55fe421594cd51754ae4215e"
format_stamp: "Formatted at 2026-08-08 03:02:28 on dist-test-slave-6ntk"
server_key: "fbbab5d512278fea015163b307f51bd3"
server_key_iv: "a8bebb7de8ac71f27a21e33e1ba90124"
server_key_version: "encryptionkey@0"
I20260808 03:02:28.504372 30419 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:02:28.508145 30436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:28.509568 30419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:02:28.509754 30419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/wal
uuid: "f059ce1b55fe421594cd51754ae4215e"
format_stamp: "Formatted at 2026-08-08 03:02:28 on dist-test-slave-6ntk"
server_key: "fbbab5d512278fea015163b307f51bd3"
server_key_iv: "a8bebb7de8ac71f27a21e33e1ba90124"
server_key_version: "encryptionkey@0"
I20260808 03:02:28.510039 30419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:28.531791 30419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:28.532781 30419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:28.533113 30419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:28.534235 30419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:28.535766 30419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:28.535884 30419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:28.536012 30419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:28.536105 30419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:28.570627 30419 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.129:34201
I20260808 03:02:28.570684 30613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.129:34201 every 8 connection(s)
I20260808 03:02:28.571973 30419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/data/info.pb
I20260808 03:02:28.575450 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 30419
I20260808 03:02:28.575650 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/wal/instance
I20260808 03:02:28.576243 822 external_mini_cluster.cc:1428] Setting key d1909fff380da5c02b7b49992ddf31f9
I20260808 03:02:28.579569 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.130:0
--local_ip_for_outbound_sockets=127.0.205.130
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:40233
--builtin_ntp_servers=127.0.205.148:33039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:02:28.586860 30619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:28.574681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:53663 (local address 127.0.205.190:40233)
0808 03:02:28.575174 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:28.575181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:28.576660 (+ 1479us) server_negotiation.cc:407] Connection header received
0808 03:02:28.578966 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:28.578970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:28.579042 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:28.579130 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:28.580605 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:28.581310 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:28.583861 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:28.584265 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:28.584365 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:28.586092 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:28.586096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:28.586099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:28.586117 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:28.586157 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:28.586171 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:28.586245 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:28.586506 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 03:02:28.586594 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":242,"threads_started":1}
I20260808 03:02:28.588389 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:28.575352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:40233 (local address 127.0.205.129:53663)
0808 03:02:28.576475 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:28.576525 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:28.578661 (+ 2136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:28.579417 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:28.579430 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:28.579547 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:28.580423 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:28.580452 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:28.581481 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:28.581490 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:28.583716 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:28.583734 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:28.584493 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:28.584502 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:28.584846 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:28.585669 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:02:28.585675 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:28.585732 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:02:28.585849 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:28.586270 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:28.586283 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:28.586293 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 03:02:28.586480 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 03:02:28.586943 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":322,"threads_started":1}
I20260808 03:02:28.589974 30614 heartbeater.cc:347] Connected to a master server at 127.0.205.190:40233
I20260808 03:02:28.590312 30614 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:28.591307 30614 heartbeater.cc:511] Master 127.0.205.190:40233 requested a full tablet report, sending...
I20260808 03:02:28.594002 30318 ts_manager.cc:194] Registered new tserver with Master: f059ce1b55fe421594cd51754ae4215e (127.0.205.129:34201)
I20260808 03:02:28.595714 30318 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.129:53663
W20260808 03:02:28.937512 30621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:28.937810 30621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:28.937992 30621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:28.945580 30621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:28.945688 30621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:28.945855 30621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.130
I20260808 03:02:28.955403 30621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:33039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.205.130
--webserver_port=0
--tserver_master_addrs=127.0.205.190:40233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.130
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:28.956776 30621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:28.958845 30621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:28.972908 30626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:28.972867 30629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:28.974051 30631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:28.974284 30621 server_base.cc:1034] running on GCE node
I20260808 03:02:28.975018 30621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:28.976095 30621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:28.977334 30621 hybrid_clock.cc:648] HybridClock initialized: now 1786158148977275 us; error 49 us; skew 500 ppm
I20260808 03:02:28.977689 30621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:28.980247 30621 webserver.cc:460] Webserver started at http://127.0.205.130:43973/ using document root <none> and password file <none>
I20260808 03:02:28.981010 30621 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:28.981150 30621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:28.981508 30621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:28.984616 30621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/data/instance:
uuid: "f407d1dc68484442ae6f0c9c30afeb75"
format_stamp: "Formatted at 2026-08-08 03:02:28 on dist-test-slave-6ntk"
server_key: "8af31bd040b705376fb51405c66f56e6"
server_key_iv: "a19ba8d240b0629a965aebbe07affbd9"
server_key_version: "encryptionkey@0"
I20260808 03:02:28.985455 30621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/wal/instance:
uuid: "f407d1dc68484442ae6f0c9c30afeb75"
format_stamp: "Formatted at 2026-08-08 03:02:28 on dist-test-slave-6ntk"
server_key: "8af31bd040b705376fb51405c66f56e6"
server_key_iv: "a19ba8d240b0629a965aebbe07affbd9"
server_key_version: "encryptionkey@0"
I20260808 03:02:28.990908 30621 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 03:02:28.994799 30638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:28.996299 30621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:28.996501 30621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/wal
uuid: "f407d1dc68484442ae6f0c9c30afeb75"
format_stamp: "Formatted at 2026-08-08 03:02:28 on dist-test-slave-6ntk"
server_key: "8af31bd040b705376fb51405c66f56e6"
server_key_iv: "a19ba8d240b0629a965aebbe07affbd9"
server_key_version: "encryptionkey@0"
I20260808 03:02:28.996742 30621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:29.021284 30621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:29.022295 30621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:29.022614 30621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:29.023656 30621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:29.025182 30621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:29.025310 30621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:29.025449 30621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:29.025535 30621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:29.060565 30621 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.130:44535
I20260808 03:02:29.060654 30816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.130:44535 every 8 connection(s)
I20260808 03:02:29.061946 30621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/data/info.pb
I20260808 03:02:29.067653 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 30621
I20260808 03:02:29.067849 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/wal/instance
I20260808 03:02:29.068432 822 external_mini_cluster.cc:1428] Setting key a0d931fa6a9d2f1d459f3e2fec457ccc
I20260808 03:02:29.071599 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.205.131:0
--local_ip_for_outbound_sockets=127.0.205.131
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:40233
--builtin_ntp_servers=127.0.205.148:33039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 03:02:29.076894 30619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.064798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.130:39021 (local address 127.0.205.190:40233)
0808 03:02:29.065013 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:29.065018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:29.066504 (+ 1486us) server_negotiation.cc:407] Connection header received
0808 03:02:29.068892 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:29.068897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:29.068971 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:29.069100 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:29.070241 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.071122 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.073750 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.074331 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.074477 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.076164 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:29.076169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:29.076173 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:29.076192 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:29.076224 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:29.076236 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:29.076302 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:29.076564 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 03:02:29.076645 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 03:02:29.078482 30821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.065345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:40233 (local address 127.0.205.130:39021)
0808 03:02:29.066317 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:29.066383 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:29.068567 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:29.069302 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:29.069310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:29.069418 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:29.070069 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.070089 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.071301 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.071309 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.073590 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.073609 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.074600 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.074609 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.074962 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.075771 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:02:29.075776 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:29.075832 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:02:29.075953 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:29.076326 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:29.076339 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:29.076350 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 03:02:29.076551 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 03:02:29.077001 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":276,"threads_started":1}
I20260808 03:02:29.079957 30817 heartbeater.cc:347] Connected to a master server at 127.0.205.190:40233
I20260808 03:02:29.080317 30817 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:29.081285 30817 heartbeater.cc:511] Master 127.0.205.190:40233 requested a full tablet report, sending...
I20260808 03:02:29.082988 30318 ts_manager.cc:194] Registered new tserver with Master: f407d1dc68484442ae6f0c9c30afeb75 (127.0.205.130:44535)
I20260808 03:02:29.083827 30318 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.130:39021
W20260808 03:02:29.432889 30822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:29.433209 30822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:29.433301 30822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:29.441236 30822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:29.441359 30822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:29.441560 30822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.205.131
I20260808 03:02:29.451097 30822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:33039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.205.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.205.131
--webserver_port=0
--tserver_master_addrs=127.0.205.190:40233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.205.131
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:29.452593 30822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:29.454604 30822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:29.469142 30834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:29.469187 30829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:29.469245 30830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:29.470216 30822 server_base.cc:1034] running on GCE node
I20260808 03:02:29.470961 30822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:29.472039 30822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:29.473321 30822 hybrid_clock.cc:648] HybridClock initialized: now 1786158149473267 us; error 58 us; skew 500 ppm
I20260808 03:02:29.473687 30822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 03:02:29.476191 30822 webserver.cc:460] Webserver started at http://127.0.205.131:36021/ using document root <none> and password file <none>
I20260808 03:02:29.476948 30822 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:29.477083 30822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:29.477415 30822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:29.480423 30822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/data/instance:
uuid: "3740812081e54a9f893da6c1ef4b82fe"
format_stamp: "Formatted at 2026-08-08 03:02:29 on dist-test-slave-6ntk"
server_key: "7f6cb280f0c99b9bd26c5c03298c6a7d"
server_key_iv: "60e0ce8a0126332bf53506fc09b60d89"
server_key_version: "encryptionkey@0"
I20260808 03:02:29.481227 30822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/wal/instance:
uuid: "3740812081e54a9f893da6c1ef4b82fe"
format_stamp: "Formatted at 2026-08-08 03:02:29 on dist-test-slave-6ntk"
server_key: "7f6cb280f0c99b9bd26c5c03298c6a7d"
server_key_iv: "60e0ce8a0126332bf53506fc09b60d89"
server_key_version: "encryptionkey@0"
I20260808 03:02:29.486647 30822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 03:02:29.490201 30843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:29.491523 30822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:02:29.491696 30822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/wal
uuid: "3740812081e54a9f893da6c1ef4b82fe"
format_stamp: "Formatted at 2026-08-08 03:02:29 on dist-test-slave-6ntk"
server_key: "7f6cb280f0c99b9bd26c5c03298c6a7d"
server_key_iv: "60e0ce8a0126332bf53506fc09b60d89"
server_key_version: "encryptionkey@0"
I20260808 03:02:29.491922 30822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:29.514840 30822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:29.515790 30822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:29.516122 30822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:29.517217 30822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 03:02:29.518957 30822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 03:02:29.519078 30822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:29.519212 30822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 03:02:29.519297 30822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:29.553287 30822 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.131:46119
I20260808 03:02:29.553382 31022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.131:46119 every 8 connection(s)
I20260808 03:02:29.554679 30822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/data/info.pb
I20260808 03:02:29.556659 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 30822
I20260808 03:02:29.556857 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/wal/instance
I20260808 03:02:29.557479 822 external_mini_cluster.cc:1428] Setting key 554698aadae3b1b1f846762903a64057
I20260808 03:02:29.572999 30619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.557427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.131:36711 (local address 127.0.205.190:40233)
0808 03:02:29.557623 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:29.557630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:29.559436 (+ 1806us) server_negotiation.cc:407] Connection header received
0808 03:02:29.561595 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:29.561600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:29.561669 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:29.561751 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:29.562822 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.563628 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.565115 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.565509 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.565600 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.571842 (+ 6242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:29.571849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:29.571853 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:29.571876 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:29.571934 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:29.571954 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:29.572062 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:29.572480 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 03:02:29.572588 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 03:02:29.573974 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.558104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:40233 (local address 127.0.205.131:36711)
0808 03:02:29.559248 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:29.559310 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:29.561348 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:29.562008 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:29.562017 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:29.562108 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:02:29.562670 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.562687 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.563771 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.563775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.564998 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.565010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.565689 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.565694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.565972 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.571456 (+ 5484us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 03:02:29.571460 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:02:29.571519 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 03:02:29.571639 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:02:29.572078 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:02:29.572091 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:02:29.572099 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 03:02:29.572280 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 03:02:29.572728 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":369,"threads_started":1}
I20260808 03:02:29.575438 31024 heartbeater.cc:347] Connected to a master server at 127.0.205.190:40233
I20260808 03:02:29.575742 31024 heartbeater.cc:464] Registering TS with master...
I20260808 03:02:29.576469 31024 heartbeater.cc:511] Master 127.0.205.190:40233 requested a full tablet report, sending...
I20260808 03:02:29.577793 30318 ts_manager.cc:194] Registered new tserver with Master: 3740812081e54a9f893da6c1ef4b82fe (127.0.205.131:46119)
I20260808 03:02:29.578622 30318 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.205.131:36711
I20260808 03:02:29.585482 822 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 03:02:29.595721 30619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.589698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.0.205.190:40233)
0808 03:02:29.589973 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:29.589981 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:29.590306 (+ 325us) server_negotiation.cc:407] Connection header received
0808 03:02:29.590447 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:29.590451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:29.590529 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:29.590614 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:29.591093 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.591780 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.593124 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.593840 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.594027 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.595065 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:29.595073 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:29.595080 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:29.595105 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:29.595149 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:29.595168 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:29.595264 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:29.595361 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 03:02:29.595485 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:02:29.599231 30614 heartbeater.cc:503] Master 127.0.205.190:40233 was elected leader, sending a full tablet report...
I20260808 03:02:29.600155 30318 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 03:02:29.607455 30318 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 03:02:29.632505 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.620653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44634 (local address 127.0.205.130:44535)
0808 03:02:29.621281 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:29.621291 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:29.621500 (+ 209us) server_negotiation.cc:407] Connection header received
0808 03:02:29.621809 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:29.621819 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:29.622144 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:29.622599 (+ 455us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:29.625506 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.626996 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.629972 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.631335 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.631460 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.631732 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:29.631934 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 03:02:29.632128 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":338,"threads_started":1}
I20260808 03:02:29.632643 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.620470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:44535 (local address 127.0.0.1:44634)
0808 03:02:29.621340 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:29.621385 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:29.621599 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:29.622399 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:29.622408 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:29.624682 (+ 2274us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x61200002c1a8. stack: 00007f5b35f9066e 00007f5b35f90430 00007f5b350175ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f5b35f860fd 00007f5b35f85d1e 00007f5b48e8f3cd 00007f5b48e8f36c 00007f5b48e5bb3b 00007f5b3c9978e9 00007f5b3c993e45 00007f5b3ca1bdca 00007f5b3ca19f68 00007f5b3ca4d82f
0808 03:02:29.624715 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:29.625328 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.625345 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.627229 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.627235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.629753 (+ 2518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.629768 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.631454 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.631460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.631666 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.631723 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 03:02:29.631867 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 03:02:29.632173 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":134,"spinlock_wait_cycles":3587456,"thread_start_us":395,"threads_started":1}
I20260808 03:02:29.633615 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.622097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36006 (local address 127.0.205.131:46119)
0808 03:02:29.623427 (+ 1330us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:29.623450 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:29.623494 (+ 44us) server_negotiation.cc:407] Connection header received
0808 03:02:29.624232 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:29.624241 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:29.624506 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:29.624720 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:29.626002 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.627602 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.629776 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.631271 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.631442 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.631765 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:29.633049 (+ 1284us) server_negotiation.cc:299] Negotiation successful
0808 03:02:29.633226 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1114,"thread_start_us":940,"threads_started":1}
I20260808 03:02:29.633682 31046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.622544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.129:34201 (local address 127.0.0.1:34566)
0808 03:02:29.625120 (+ 2576us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:29.625140 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:29.625262 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:29.626054 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:29.626059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:29.626084 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:29.626391 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.626401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.628264 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.628269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.630480 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.630495 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.632840 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.632848 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.633096 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.633131 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 03:02:29.633226 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 03:02:29.633359 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1270,"mutex_wait_us":3,"thread_start_us":1102,"threads_started":1}
I20260808 03:02:29.633919 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.622633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34566 (local address 127.0.205.129:34201)
0808 03:02:29.625425 (+ 2792us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:29.625434 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:29.625461 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:02:29.625582 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:29.625593 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:29.625815 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:29.626069 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:29.626562 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.628104 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.630721 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.632282 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.632427 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.632706 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:29.633315 (+ 609us) server_negotiation.cc:299] Negotiation successful
0808 03:02:29.633494 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2607,"thread_start_us":2467,"threads_started":1}
I20260808 03:02:29.634420 31043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.621673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:46119 (local address 127.0.0.1:36006)
0808 03:02:29.622237 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:29.622257 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:29.624020 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:29.624720 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:29.624725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:29.624751 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:29.625731 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.625742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.627780 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.627785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.629571 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.629584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.631921 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.631928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.632153 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.632183 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 03:02:29.632274 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 03:02:29.632414 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":8,"spinlock_wait_cycles":7040,"thread_start_us":283,"threads_started":1}
I20260808 03:02:29.637456 30508 tablet_service.cc:1467] Processing CreateTablet for tablet f429270813594a64a3295edf92a2bd2b (DEFAULT_TABLE table=test-table [id=7457fb687a434fa098723020aedbfb78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:29.637423 30719 tablet_service.cc:1467] Processing CreateTablet for tablet f429270813594a64a3295edf92a2bd2b (DEFAULT_TABLE table=test-table [id=7457fb687a434fa098723020aedbfb78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:29.637840 30928 tablet_service.cc:1467] Processing CreateTablet for tablet f429270813594a64a3295edf92a2bd2b (DEFAULT_TABLE table=test-table [id=7457fb687a434fa098723020aedbfb78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 03:02:29.638852 30508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f429270813594a64a3295edf92a2bd2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:29.638916 30719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f429270813594a64a3295edf92a2bd2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:29.639230 30928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f429270813594a64a3295edf92a2bd2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:29.651536 31048 tablet_bootstrap.cc:492] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: Bootstrap starting.
I20260808 03:02:29.654620 31050 tablet_bootstrap.cc:492] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: Bootstrap starting.
I20260808 03:02:29.657231 31048 tablet_bootstrap.cc:654] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:29.657780 31049 tablet_bootstrap.cc:492] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: Bootstrap starting.
I20260808 03:02:29.658283 31050 tablet_bootstrap.cc:654] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:29.658622 31048 log.cc:824] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:29.659543 31050 log.cc:824] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:29.661124 31049 tablet_bootstrap.cc:654] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:29.662199 31049 log.cc:824] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:29.662413 31048 tablet_bootstrap.cc:492] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: No bootstrap required, opened a new log
I20260808 03:02:29.662742 31048 ts_tablet_manager.cc:1397] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260808 03:02:29.666100 31050 tablet_bootstrap.cc:492] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: No bootstrap required, opened a new log
I20260808 03:02:29.666393 31050 ts_tablet_manager.cc:1397] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260808 03:02:29.668473 31049 tablet_bootstrap.cc:492] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: No bootstrap required, opened a new log
I20260808 03:02:29.668780 31049 ts_tablet_manager.cc:1397] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260808 03:02:29.673470 31048 raft_consensus.cc:348] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 } } peers { permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 } } peers { permanent_uuid: "f059ce1b55fe421594cd51754ae4215e" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34201 } }
I20260808 03:02:29.674015 31048 raft_consensus.cc:374] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:29.674130 31048 raft_consensus.cc:729] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f059ce1b55fe421594cd51754ae4215e, State: Initialized, Role: FOLLOWER
I20260808 03:02:29.674875 31048 consensus_queue.cc:260] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 } } peers { permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 } } peers { permanent_uuid: "f059ce1b55fe421594cd51754ae4215e" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34201 } }
I20260808 03:02:29.676530 31049 raft_consensus.cc:348] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 } } peers { permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 } } peers { permanent_uuid: "f059ce1b55fe421594cd51754ae4215e" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34201 } }
I20260808 03:02:29.677019 31049 raft_consensus.cc:374] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:29.677112 31049 raft_consensus.cc:729] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f407d1dc68484442ae6f0c9c30afeb75, State: Initialized, Role: FOLLOWER
I20260808 03:02:29.677114 31050 raft_consensus.cc:348] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 } } peers { permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 } } peers { permanent_uuid: "f059ce1b55fe421594cd51754ae4215e" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34201 } }
I20260808 03:02:29.677495 31048 ts_tablet_manager.cc:1428] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260808 03:02:29.677577 31050 raft_consensus.cc:374] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:29.677666 31050 raft_consensus.cc:729] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3740812081e54a9f893da6c1ef4b82fe, State: Initialized, Role: FOLLOWER
I20260808 03:02:29.677809 31049 consensus_queue.cc:260] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 } } peers { permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 } } peers { permanent_uuid: "f059ce1b55fe421594cd51754ae4215e" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34201 } }
I20260808 03:02:29.678277 31050 consensus_queue.cc:260] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 } } peers { permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 } } peers { permanent_uuid: "f059ce1b55fe421594cd51754ae4215e" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34201 } }
I20260808 03:02:29.679234 30817 heartbeater.cc:503] Master 127.0.205.190:40233 was elected leader, sending a full tablet report...
I20260808 03:02:29.679944 31049 ts_tablet_manager.cc:1428] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260808 03:02:29.681211 31024 heartbeater.cc:503] Master 127.0.205.190:40233 was elected leader, sending a full tablet report...
I20260808 03:02:29.681962 31050 ts_tablet_manager.cc:1428] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260808 03:02:29.806466 31025 tablet.cc:2369] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:02:29.814152 30818 tablet.cc:2369] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 03:02:29.824478 30617 tablet.cc:2369] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:02:29.937235 31055 raft_consensus.cc:482] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 03:02:29.937564 31055 raft_consensus.cc:504] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 } } peers { permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 } } peers { permanent_uuid: "f059ce1b55fe421594cd51754ae4215e" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34201 } }
I20260808 03:02:29.939306 31055 leader_election.cc:288] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f407d1dc68484442ae6f0c9c30afeb75 (127.0.205.130:44535), 3740812081e54a9f893da6c1ef4b82fe (127.0.205.131:46119)
I20260808 03:02:29.945981 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.939758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:50181 (local address 127.0.205.130:44535)
0808 03:02:29.940036 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:29.940045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:29.940515 (+ 470us) server_negotiation.cc:407] Connection header received
0808 03:02:29.940785 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:29.940795 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:29.940927 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:29.941040 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:29.941737 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.942518 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.944295 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.945038 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.945203 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.945277 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:29.945550 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 03:02:29.945679 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:02:29.946019 31060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.939733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.130:44535 (local address 127.0.205.129:50181)
0808 03:02:29.940376 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:29.940404 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:29.940592 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:29.941092 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:29.941102 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:29.941137 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:29.941581 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.941594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.942649 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.942657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.944100 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.944113 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.945230 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.945236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.945392 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.945417 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 03:02:29.945520 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 03:02:29.945647 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":359,"threads_started":1}
I20260808 03:02:29.946427 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.940711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.205.129:58049 (local address 127.0.205.131:46119)
0808 03:02:29.940908 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:29.940917 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:29.941441 (+ 524us) server_negotiation.cc:407] Connection header received
0808 03:02:29.941613 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:29.941616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:29.941674 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:29.941779 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 03:02:29.942332 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.943014 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.944719 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.945498 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.945698 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.945769 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:29.946071 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 03:02:29.946197 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:02:29.946489 31061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.940566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.131:46119 (local address 127.0.205.129:58049)
0808 03:02:29.941325 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0808 03:02:29.941346 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 03:02:29.941484 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:02:29.941799 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:02:29.941803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:02:29.941827 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 03:02:29.942170 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.942179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.943156 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.943162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.944540 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:02:29.944553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.945692 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:02:29.945699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:02:29.945859 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:29.945937 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 03:02:29.946046 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 03:02:29.946167 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":386,"threads_started":1}
I20260808 03:02:29.947149 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f429270813594a64a3295edf92a2bd2b" candidate_uuid: "f059ce1b55fe421594cd51754ae4215e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f407d1dc68484442ae6f0c9c30afeb75" is_pre_election: true
I20260808 03:02:29.947463 30957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f429270813594a64a3295edf92a2bd2b" candidate_uuid: "f059ce1b55fe421594cd51754ae4215e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3740812081e54a9f893da6c1ef4b82fe" is_pre_election: true
I20260808 03:02:29.947693 30747 raft_consensus.cc:2454] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f059ce1b55fe421594cd51754ae4215e in term 0.
I20260808 03:02:29.947976 30957 raft_consensus.cc:2454] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f059ce1b55fe421594cd51754ae4215e in term 0.
I20260808 03:02:29.948458 30437 leader_election.cc:302] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f059ce1b55fe421594cd51754ae4215e, f407d1dc68484442ae6f0c9c30afeb75; no voters:
I20260808 03:02:29.948839 31055 raft_consensus.cc:2781] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 03:02:29.949005 31055 raft_consensus.cc:482] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 03:02:29.949121 31055 raft_consensus.cc:3037] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:29.951427 31055 raft_consensus.cc:504] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 } } peers { permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 } } peers { permanent_uuid: "f059ce1b55fe421594cd51754ae4215e" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34201 } }
I20260808 03:02:29.952062 31055 leader_election.cc:288] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [CANDIDATE]: Term 1 election: Requested vote from peers f407d1dc68484442ae6f0c9c30afeb75 (127.0.205.130:44535), 3740812081e54a9f893da6c1ef4b82fe (127.0.205.131:46119)
I20260808 03:02:29.952481 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f429270813594a64a3295edf92a2bd2b" candidate_uuid: "f059ce1b55fe421594cd51754ae4215e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f407d1dc68484442ae6f0c9c30afeb75"
I20260808 03:02:29.952574 30957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f429270813594a64a3295edf92a2bd2b" candidate_uuid: "f059ce1b55fe421594cd51754ae4215e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3740812081e54a9f893da6c1ef4b82fe"
I20260808 03:02:29.952675 30747 raft_consensus.cc:3037] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:29.952814 30957 raft_consensus.cc:3037] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:29.954995 30747 raft_consensus.cc:2454] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f059ce1b55fe421594cd51754ae4215e in term 1.
I20260808 03:02:29.955060 30957 raft_consensus.cc:2454] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f059ce1b55fe421594cd51754ae4215e in term 1.
I20260808 03:02:29.955446 30437 leader_election.cc:302] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f059ce1b55fe421594cd51754ae4215e, f407d1dc68484442ae6f0c9c30afeb75; no voters:
I20260808 03:02:29.955740 31055 raft_consensus.cc:2781] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:29.956177 31055 raft_consensus.cc:686] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 1 LEADER]: Becoming Leader. State: Replica: f059ce1b55fe421594cd51754ae4215e, State: Running, Role: LEADER
I20260808 03:02:29.956643 31055 consensus_queue.cc:237] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 } } peers { permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 } } peers { permanent_uuid: "f059ce1b55fe421594cd51754ae4215e" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34201 } }
I20260808 03:02:29.961740 30317 catalog_manager.cc:5495] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e reported cstate change: term changed from 0 to 1, leader changed from <none> to f059ce1b55fe421594cd51754ae4215e (127.0.205.129). New cstate: current_term: 1 leader_uuid: "f059ce1b55fe421594cd51754ae4215e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f059ce1b55fe421594cd51754ae4215e" member_type: VOTER last_known_addr { host: "127.0.205.129" port: 34201 } health_report { overall_health: HEALTHY } } }
I20260808 03:02:30.003841 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:29.998059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.0.205.129:34201)
0808 03:02:29.998278 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:29.998285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:29.998388 (+ 103us) server_negotiation.cc:407] Connection header received
0808 03:02:29.998618 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:29.998622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:29.998720 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:29.998856 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:02:29.999209 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:29.999909 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:30.001581 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:30.002088 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:30.002190 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:30.002760 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:02:30.003325 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:02:30.003459 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:30.003567 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 03:02:30.003642 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 03:02:30.012406 30957 raft_consensus.cc:1261] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe [term 1 FOLLOWER]: Refusing update from remote peer f059ce1b55fe421594cd51754ae4215e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:30.012429 30747 raft_consensus.cc:1261] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75 [term 1 FOLLOWER]: Refusing update from remote peer f059ce1b55fe421594cd51754ae4215e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:02:30.013396 31063 consensus_queue.cc:1035] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3740812081e54a9f893da6c1ef4b82fe" member_type: VOTER last_known_addr { host: "127.0.205.131" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:30.013788 31055 consensus_queue.cc:1035] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f407d1dc68484442ae6f0c9c30afeb75" member_type: VOTER last_known_addr { host: "127.0.205.130" port: 44535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:02:30.043169 31074 mvcc.cc:205] Tried to move back new op lower bound from 7316103782440837120 to 7316103782226313216. Current Snapshot: MvccSnapshot[applied={T|T < 7316103782440837120}]
I20260808 03:02:30.045087 30317 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:30.045420 30317 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 03:02:30.049899 30317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36819fcb1efd47a39209e8928dd533f6: Sending DeleteTablet for 3 replicas of tablet f429270813594a64a3295edf92a2bd2b
I20260808 03:02:30.051329 30928 tablet_service.cc:1514] Processing DeleteTablet for tablet f429270813594a64a3295edf92a2bd2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:30 UTC) from {username='slave'} at 127.0.0.1:36006
I20260808 03:02:30.051707 30719 tablet_service.cc:1514] Processing DeleteTablet for tablet f429270813594a64a3295edf92a2bd2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:30 UTC) from {username='slave'} at 127.0.0.1:44634
I20260808 03:02:30.051608 30718 tablet_service.cc:1514] Processing DeleteTablet for tablet f429270813594a64a3295edf92a2bd2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:30 UTC) from {username='slave'} at 127.0.0.1:44634
I20260808 03:02:30.052390 30508 tablet_service.cc:1514] Processing DeleteTablet for tablet f429270813594a64a3295edf92a2bd2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 03:02:30 UTC) from {username='slave'} at 127.0.0.1:34566
I20260808 03:02:30.052642 31086 tablet_replica.cc:330] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: stopping tablet replica
I20260808 03:02:30.053177 31086 raft_consensus.cc:2229] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:30.053737 31086 raft_consensus.cc:2258] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:30.054674 31087 tablet_replica.cc:330] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: stopping tablet replica
I20260808 03:02:30.054872 31085 tablet_replica.cc:330] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: stopping tablet replica
I20260808 03:02:30.055197 31087 raft_consensus.cc:2229] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 1 LEADER]: Raft consensus shutting down.
I20260808 03:02:30.055327 31085 raft_consensus.cc:2229] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 03:02:30.055971 31085 raft_consensus.cc:2258] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:30.056003 31087 raft_consensus.cc:2258] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 03:02:30.058200 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 30419
I20260808 03:02:30.058640 31085 ts_tablet_manager.cc:1905] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:30.058712 31087 ts_tablet_manager.cc:1905] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:30.069094 31087 ts_tablet_manager.cc:1918] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:30.069342 31087 log.cc:1196] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-0/wal/wals/f429270813594a64a3295edf92a2bd2b
I20260808 03:02:30.070041 31087 ts_tablet_manager.cc:1939] T f429270813594a64a3295edf92a2bd2b P f059ce1b55fe421594cd51754ae4215e: Deleting consensus metadata
I20260808 03:02:30.070041 31086 ts_tablet_manager.cc:1905] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 03:02:30.072161 30295 catalog_manager.cc:4841] TS f059ce1b55fe421594cd51754ae4215e (127.0.205.129:34201): tablet f429270813594a64a3295edf92a2bd2b (table test-table [id=7457fb687a434fa098723020aedbfb78]) successfully deleted
I20260808 03:02:30.078413 31085 ts_tablet_manager.cc:1918] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:30.078702 31085 log.cc:1196] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-2/wal/wals/f429270813594a64a3295edf92a2bd2b
I20260808 03:02:30.079339 31085 ts_tablet_manager.cc:1939] T f429270813594a64a3295edf92a2bd2b P 3740812081e54a9f893da6c1ef4b82fe: Deleting consensus metadata
I20260808 03:02:30.081377 30295 catalog_manager.cc:4841] TS 3740812081e54a9f893da6c1ef4b82fe (127.0.205.131:46119): tablet f429270813594a64a3295edf92a2bd2b (table test-table [id=7457fb687a434fa098723020aedbfb78]) successfully deleted
I20260808 03:02:30.085299 31086 ts_tablet_manager.cc:1918] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 03:02:30.085608 31086 log.cc:1196] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: Deleting WAL directory at /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786157811617362-822-0/minicluster-data/ts-1/wal/wals/f429270813594a64a3295edf92a2bd2b
I20260808 03:02:30.086326 31086 ts_tablet_manager.cc:1939] T f429270813594a64a3295edf92a2bd2b P f407d1dc68484442ae6f0c9c30afeb75: Deleting consensus metadata
I20260808 03:02:30.087265 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:30.060548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34592 (local address 127.0.205.129:34201)
0808 03:02:30.060748 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:30.060755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:30.078179 (+ 17424us) server_negotiation.cc:407] Connection header received
0808 03:02:30.078652 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:30.078658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:30.078760 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:30.078833 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:30.079509 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:30.080680 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:30.082613 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:30.083314 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:30.083433 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:30.085422 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:30.085434 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:30.085455 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:30.086323 (+ 868us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:30.086400 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:30.086430 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:30.086516 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:30.086773 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 03:02:30.086896 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 03:02:30.088578 30604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:30.088604 30295 catalog_manager.cc:4841] TS f407d1dc68484442ae6f0c9c30afeb75 (127.0.205.130:44535): tablet f429270813594a64a3295edf92a2bd2b (table test-table [id=7457fb687a434fa098723020aedbfb78]) successfully deleted
W20260808 03:02:30.089529 30295 catalog_manager.cc:4808] TS f407d1dc68484442ae6f0c9c30afeb75 (127.0.205.130:44535): delete failed for tablet f429270813594a64a3295edf92a2bd2b because the tablet was not found. No further retry: Not found: Tablet not found: f429270813594a64a3295edf92a2bd2b
I20260808 03:02:30.196411 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 30419
I20260808 03:02:30.224526 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 30621
I20260808 03:02:30.231172 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:30.225344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44638 (local address 127.0.205.130:44535)
0808 03:02:30.225520 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:30.225524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:30.225543 (+ 19us) server_negotiation.cc:407] Connection header received
0808 03:02:30.225668 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:30.225671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:30.225761 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:30.225844 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:30.226302 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:30.227033 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:30.228162 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:30.228563 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:30.228653 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:30.229621 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:30.229630 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:30.229645 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:30.230418 (+ 773us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:30.230467 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:30.230495 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:30.230583 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:30.230697 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 03:02:30.230785 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 03:02:30.231829 30810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:30.323298 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 30621
I20260808 03:02:30.348590 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 30822
I20260808 03:02:30.355443 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:30.349443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.0.205.131:46119)
0808 03:02:30.349619 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:30.349623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:30.349641 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:02:30.349802 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:30.349806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:30.349954 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:30.350042 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:30.350448 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:30.351172 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:30.352438 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:30.352822 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:30.352918 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:30.354076 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:30.354086 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:30.354102 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:30.354686 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:30.354731 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:30.354751 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:30.354858 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:30.355095 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 03:02:30.355187 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 03:02:30.356094 31017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:30.445791 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 30822
I20260808 03:02:30.468112 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 30274
I20260808 03:02:30.469173 30369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:30.582466 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 30274
2026-08-08T03:02:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3182 ms)
[----------] 6 tests from EncryptionPolicyTest (24560 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:30 dist-test-slave-6ntk krb5kdc[31101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:30 dist-test-slave-6ntk krb5kdc[31101](info): set up 2 sockets
Aug 08 03:02:30 dist-test-slave-6ntk krb5kdc[31101](info): commencing operation
krb5kdc: starting...
W20260808 03:02:31.972640 822 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:32 dist-test-slave-6ntk krb5kdc[31101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:32.317859 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:32.683219 31118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:32.683610 31118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:32.683768 31118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:32.691527 31118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:32.691699 31118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:32.691805 31118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:32.691907 31118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:32.691982 31118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:32.702554 31118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:32.704035 31118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:32.705431 31118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:32.718082 31134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:32.717988 31128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:32.718022 31126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:32.718932 31118 server_base.cc:1034] running on GCE node
I20260808 03:02:32.719831 31118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:32.721319 31118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:32.722571 31118 hybrid_clock.cc:648] HybridClock initialized: now 1786158152722511 us; error 67 us; skew 500 ppm
I20260808 03:02:32.722947 31118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:32 dist-test-slave-6ntk krb5kdc[31101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158152, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:32.726385 31118 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:32.727957 31118 webserver.cc:460] Webserver started at http://127.0.0.1:43027/ using document root <none> and password file <none>
I20260808 03:02:32.728683 31118 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:32.728833 31118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:32.729151 31118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:32.732296 31118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "95f5cf1301974f1e92bf22e3ccf90540"
format_stamp: "Formatted at 2026-08-08 03:02:32 on dist-test-slave-6ntk"
server_key: "fc2f2647400e74bd7973f0fb15429cf8"
server_key_iv: "d8d74625419e0da887938636db89c9ec"
server_key_version: "encryptionkey@0"
I20260808 03:02:32.733108 31118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "95f5cf1301974f1e92bf22e3ccf90540"
format_stamp: "Formatted at 2026-08-08 03:02:32 on dist-test-slave-6ntk"
server_key: "fc2f2647400e74bd7973f0fb15429cf8"
server_key_iv: "d8d74625419e0da887938636db89c9ec"
server_key_version: "encryptionkey@0"
I20260808 03:02:32.738484 31118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 03:02:32.742285 31143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:32.743611 31118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 03:02:32.743805 31118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "95f5cf1301974f1e92bf22e3ccf90540"
format_stamp: "Formatted at 2026-08-08 03:02:32 on dist-test-slave-6ntk"
server_key: "fc2f2647400e74bd7973f0fb15429cf8"
server_key_iv: "d8d74625419e0da887938636db89c9ec"
server_key_version: "encryptionkey@0"
I20260808 03:02:32.744011 31118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:32.761749 31118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:32.766378 31118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:32.766669 31118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:32.785918 31118 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41225
I20260808 03:02:32.785977 31217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41225 every 8 connection(s)
I20260808 03:02:32.787387 31118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:32.791399 31219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:32.793260 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 31118
I20260808 03:02:32.793526 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:32.794407 822 external_mini_cluster.cc:1428] Setting key d6050c6d6a245e975359dad13f68b6d2
I20260808 03:02:32.806816 31219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540: Bootstrap starting.
Aug 08 03:02:32 dist-test-slave-6ntk krb5kdc[31101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 03:02:32.811060 31219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:32.812546 31219 log.cc:824] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:32.816237 31219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540: No bootstrap required, opened a new log
I20260808 03:02:32.820115 31225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:32.796493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43987 (local address 127.0.0.1:41225)
0808 03:02:32.797345 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:32.797356 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:32.797413 (+ 57us) server_negotiation.cc:407] Connection header received
0808 03:02:32.799131 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:32.799158 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:32.799585 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:32.799911 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:32.802011 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:32.803410 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:32.805291 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:32.806088 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:32.806225 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:32.811351 (+ 5126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:32.811364 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:32.811397 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:32.811463 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:32.815585 (+ 4122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:32.816563 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:32.816570 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:32.816577 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:32.816769 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:32.817271 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:32.817275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:32.817277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:32.817815 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:32.818227 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:32.818737 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 03:02:32.819086 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":403,"threads_started":1}
I20260808 03:02:32.824538 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:32.825452 31219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f5cf1301974f1e92bf22e3ccf90540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41225 } }
I20260808 03:02:32.825830 31219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:32.826006 31219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f5cf1301974f1e92bf22e3ccf90540, State: Initialized, Role: FOLLOWER
I20260808 03:02:32.826577 31219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f5cf1301974f1e92bf22e3ccf90540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41225 } }
I20260808 03:02:32.826825 31219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:32.826984 31219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:32.827153 31219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:32.829612 31219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f5cf1301974f1e92bf22e3ccf90540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41225 } }
I20260808 03:02:32.830312 31219 leader_election.cc:302] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95f5cf1301974f1e92bf22e3ccf90540; no voters:
I20260808 03:02:32.831075 31219 leader_election.cc:288] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:32.831234 31237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:32.832417 31237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [term 1 LEADER]: Becoming Leader. State: Replica: 95f5cf1301974f1e92bf22e3ccf90540, State: Running, Role: LEADER
I20260808 03:02:32.833137 31237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f5cf1301974f1e92bf22e3ccf90540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41225 } }
I20260808 03:02:32.833511 31219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:32.836357 31240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95f5cf1301974f1e92bf22e3ccf90540. Latest consensus state: current_term: 1 leader_uuid: "95f5cf1301974f1e92bf22e3ccf90540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f5cf1301974f1e92bf22e3ccf90540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41225 } } }
I20260808 03:02:32.836336 31239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95f5cf1301974f1e92bf22e3ccf90540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f5cf1301974f1e92bf22e3ccf90540" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41225 } } }
I20260808 03:02:32.836853 31240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:32.836884 31239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:32.841672 31254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:32.847926 31254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:32.851666 31225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:32.829920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55237 (local address 127.0.0.1:41225)
0808 03:02:32.830110 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:32.830117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:32.830548 (+ 431us) server_negotiation.cc:407] Connection header received
0808 03:02:32.830750 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:32.830759 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:32.830869 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:32.830999 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:32.833209 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:32.834517 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:32.836407 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:32.837116 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:32.837268 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:32.841083 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:32.841092 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:32.841101 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:32.841155 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:32.846621 (+ 5466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:32.847873 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:32.847879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:32.847883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:32.848099 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:32.848766 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:32.848771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:32.848775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:32.849333 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:32.850893 (+ 1560us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:32.851074 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 03:02:32.851273 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 03:02:32.854012 31272 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 03:02:32.859305 31254 catalog_manager.cc:1347] Generated new cluster ID: 1ea5638883094dfaa73f2b05a3b440b0
I20260808 03:02:32.859441 31254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:32.897686 31254 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:32.899122 31254 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:32.905431 31254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95f5cf1301974f1e92bf22e3ccf90540: Generated new TSK 0
I20260808 03:02:32.906486 31254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:32.918304 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 31118
I20260808 03:02:32.919850 31208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:33.016945 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 31118
W20260808 03:02:33.041803 31099 connection.cc:535] client connection to 127.0.0.1:41225 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41225): Connection reset by peer (error 104)
2026-08-08T03:02:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:33 dist-test-slave-6ntk krb5kdc[31279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:33 dist-test-slave-6ntk krb5kdc[31279](info): set up 2 sockets
Aug 08 03:02:33 dist-test-slave-6ntk krb5kdc[31279](info): commencing operation
krb5kdc: starting...
W20260808 03:02:34.435107 822 mini_kdc.cc:121] Time spent starting KDC: real 1.374s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:34 dist-test-slave-6ntk krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:34.778718 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:35.155531 31299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:35.155869 31299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:35.156080 31299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:35.163760 31299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:35.163942 31299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:35.164072 31299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:35.164193 31299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:35.164274 31299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:35.174907 31299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37497
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:35.176502 31299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:35.178002 31299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:35.191534 31307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:35.191630 31304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:35.191630 31305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:35.192894 31299 server_base.cc:1034] running on GCE node
I20260808 03:02:35.193814 31299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:35.195587 31299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:35.196830 31299 hybrid_clock.cc:648] HybridClock initialized: now 1786158155196790 us; error 58 us; skew 500 ppm
I20260808 03:02:35.197206 31299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:35 dist-test-slave-6ntk krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:35.205324 31299 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:35.207202 31299 webserver.cc:460] Webserver started at http://127.0.0.1:36199/ using document root <none> and password file <none>
I20260808 03:02:35.207978 31299 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:35.208096 31299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:35.208379 31299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:35.211736 31299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "0fe7a79dc645468aaac372c501a789be"
format_stamp: "Formatted at 2026-08-08 03:02:35 on dist-test-slave-6ntk"
server_key: "afebe6ef9e503b489e99694ae061ff99"
server_key_iv: "473a2499f2a5753f325ae3a46581a0c9"
server_key_version: "encryptionkey@0"
I20260808 03:02:35.212651 31299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "0fe7a79dc645468aaac372c501a789be"
format_stamp: "Formatted at 2026-08-08 03:02:35 on dist-test-slave-6ntk"
server_key: "afebe6ef9e503b489e99694ae061ff99"
server_key_iv: "473a2499f2a5753f325ae3a46581a0c9"
server_key_version: "encryptionkey@0"
I20260808 03:02:35.218555 31299 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260808 03:02:35.222246 31317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:35.223658 31299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:35.223856 31299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "0fe7a79dc645468aaac372c501a789be"
format_stamp: "Formatted at 2026-08-08 03:02:35 on dist-test-slave-6ntk"
server_key: "afebe6ef9e503b489e99694ae061ff99"
server_key_iv: "473a2499f2a5753f325ae3a46581a0c9"
server_key_version: "encryptionkey@0"
I20260808 03:02:35.224083 31299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:35.265700 31299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:35.270534 31299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:35.270860 31299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:35.290668 31299 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37497
I20260808 03:02:35.290688 31397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37497 every 8 connection(s)
I20260808 03:02:35.292162 31299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:35.294203 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 31299
I20260808 03:02:35.294406 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:35.294854 822 external_mini_cluster.cc:1428] Setting key 85c1ccc5b47a1162b4b34360ca4bd5b3
I20260808 03:02:35.296722 31399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:35.310688 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:35.296779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49857 (local address 127.0.0.1:37497)
0808 03:02:35.297621 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:35.297636 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:35.297689 (+ 53us) server_negotiation.cc:407] Connection header received
0808 03:02:35.299980 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:35.300010 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:35.300560 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:35.301038 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:35.301587 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:35.303703 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:35.305030 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:35.306157 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:35.306344 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:35.307591 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:35.307602 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:35.307624 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:35.308449 (+ 825us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:35.308548 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:35.308584 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:35.308757 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:35.309014 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 03:02:35.309224 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":326,"threads_started":1}
I20260808 03:02:35.312116 31399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be: Bootstrap starting.
I20260808 03:02:35.314837 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:35.315877 31399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:35.317185 31399 log.cc:824] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:35.320161 31399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be: No bootstrap required, opened a new log
I20260808 03:02:35.328091 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:35.320517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47689 (local address 127.0.0.1:37497)
0808 03:02:35.320734 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:35.320741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:35.321276 (+ 535us) server_negotiation.cc:407] Connection header received
0808 03:02:35.321451 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:35.321458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:35.321536 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:35.321648 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:35.322188 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:35.323450 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:35.324992 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:35.325797 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:35.325963 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:35.327239 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:35.327246 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:35.327250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:35.327272 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:35.327319 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:35.327344 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:35.327484 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:35.327694 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 03:02:35.327748 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:02:35.330580 31399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe7a79dc645468aaac372c501a789be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } }
I20260808 03:02:35.331012 31399 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:35.331178 31399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe7a79dc645468aaac372c501a789be, State: Initialized, Role: FOLLOWER
I20260808 03:02:35.331791 31399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe7a79dc645468aaac372c501a789be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } }
I20260808 03:02:35.332046 31399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:35.332227 31399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:35.332417 31399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:35.334826 31399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe7a79dc645468aaac372c501a789be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } }
I20260808 03:02:35.335464 31399 leader_election.cc:302] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fe7a79dc645468aaac372c501a789be; no voters:
I20260808 03:02:35.336112 31399 leader_election.cc:288] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:35.336266 31412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:35.337003 31412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [term 1 LEADER]: Becoming Leader. State: Replica: 0fe7a79dc645468aaac372c501a789be, State: Running, Role: LEADER
I20260808 03:02:35.337523 31412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe7a79dc645468aaac372c501a789be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } }
I20260808 03:02:35.338344 31399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:35.340718 31414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fe7a79dc645468aaac372c501a789be. Latest consensus state: current_term: 1 leader_uuid: "0fe7a79dc645468aaac372c501a789be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe7a79dc645468aaac372c501a789be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } } }
I20260808 03:02:35.341197 31414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:35.341038 31413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fe7a79dc645468aaac372c501a789be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe7a79dc645468aaac372c501a789be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } } }
I20260808 03:02:35.341341 31413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:35.344215 31429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:35.349754 31429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:35.359825 31429 catalog_manager.cc:1347] Generated new cluster ID: b79c9897a0024fe89568bc1d920b2897
I20260808 03:02:35.359985 31429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:35.370255 31429 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:35.372205 31429 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:35.389343 31429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fe7a79dc645468aaac372c501a789be: Generated new TSK 0
I20260808 03:02:35.390559 31429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:35.404526 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 31299
W20260808 03:02:35.406304 31389 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49857
W20260808 03:02:35.407197 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 31299: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:02:35.407409 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 31299
W20260808 03:02:35.430658 31276 connection.cc:535] client connection to 127.0.0.1:37497 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37497): Connection reset by peer (error 104)
2026-08-08T03:02:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:35 dist-test-slave-6ntk krb5kdc[31446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:35 dist-test-slave-6ntk krb5kdc[31446](info): set up 2 sockets
Aug 08 03:02:35 dist-test-slave-6ntk krb5kdc[31446](info): commencing operation
krb5kdc: starting...
W20260808 03:02:36.790019 822 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:37 dist-test-slave-6ntk krb5kdc[31446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:37.134094 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:37.515213 31468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:37.515576 31468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:37.515829 31468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:37.524390 31468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:37.524531 31468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:37.524662 31468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:37.524785 31468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:37.524849 31468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:37.535358 31468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33897
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:37.536746 31468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:37.538177 31468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:37.551095 31478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:37.550976 31482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:37.551060 31479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:37.552560 31468 server_base.cc:1034] running on GCE node
I20260808 03:02:37.553498 31468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:37.555181 31468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:37.556382 31468 hybrid_clock.cc:648] HybridClock initialized: now 1786158157556296 us; error 72 us; skew 500 ppm
I20260808 03:02:37.556805 31468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:37 dist-test-slave-6ntk krb5kdc[31446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:37.560328 31468 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:37.562038 31468 webserver.cc:460] Webserver started at http://127.0.0.1:36157/ using document root <none> and password file <none>
I20260808 03:02:37.562779 31468 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:37.562894 31468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:37.563185 31468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:37.566515 31468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "b6ee56d9ac8e441b9c6b567be5da9be2"
format_stamp: "Formatted at 2026-08-08 03:02:37 on dist-test-slave-6ntk"
server_key: "4a5a8bb9b0d712f7381ac7e99955baef"
server_key_iv: "bbd427b69a764dbaa3260b0fb419f8e8"
server_key_version: "encryptionkey@0"
I20260808 03:02:37.567338 31468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "b6ee56d9ac8e441b9c6b567be5da9be2"
format_stamp: "Formatted at 2026-08-08 03:02:37 on dist-test-slave-6ntk"
server_key: "4a5a8bb9b0d712f7381ac7e99955baef"
server_key_iv: "bbd427b69a764dbaa3260b0fb419f8e8"
server_key_version: "encryptionkey@0"
I20260808 03:02:37.573096 31468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:02:37.577140 31501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:37.578644 31468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:02:37.578855 31468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "b6ee56d9ac8e441b9c6b567be5da9be2"
format_stamp: "Formatted at 2026-08-08 03:02:37 on dist-test-slave-6ntk"
server_key: "4a5a8bb9b0d712f7381ac7e99955baef"
server_key_iv: "bbd427b69a764dbaa3260b0fb419f8e8"
server_key_version: "encryptionkey@0"
I20260808 03:02:37.579082 31468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:37.585633 31468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:37.590494 31468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:37.590850 31468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:37.612159 31468 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33897
I20260808 03:02:37.612223 31585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33897 every 8 connection(s)
I20260808 03:02:37.613842 31468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:37.618827 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 31468
I20260808 03:02:37.619045 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:37.619462 31589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:37.619681 822 external_mini_cluster.cc:1428] Setting key 6070a1939afd38dd1230edc3b37f90c5
I20260808 03:02:37.629222 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:37.621696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57121 (local address 127.0.0.1:33897)
0808 03:02:37.622672 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:37.622688 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:37.622746 (+ 58us) server_negotiation.cc:407] Connection header received
0808 03:02:37.624844 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:37.624876 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:37.625364 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:37.625919 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:37.627112 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:37.627121 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:37.627144 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:37.627167 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:37.627255 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:37.627291 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:37.627550 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:37.627696 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 03:02:37.627903 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":423,"threads_started":1}
I20260808 03:02:37.633567 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:37.635300 31589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2: Bootstrap starting.
I20260808 03:02:37.639714 31589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:37.641393 31589 log.cc:824] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:37.642298 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:37.639199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55653 (local address 127.0.0.1:33897)
0808 03:02:37.639402 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:37.639412 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:37.639796 (+ 384us) server_negotiation.cc:407] Connection header received
0808 03:02:37.639984 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:37.639989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:37.640042 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:37.640159 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:37.641382 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:37.641391 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:37.641395 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:37.641416 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:37.641465 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:37.641481 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:37.641614 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:37.641800 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 03:02:37.641850 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:02:37.644727 31589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2: No bootstrap required, opened a new log
I20260808 03:02:37.652854 31589 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ee56d9ac8e441b9c6b567be5da9be2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33897 } }
I20260808 03:02:37.653374 31589 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:37.653506 31589 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ee56d9ac8e441b9c6b567be5da9be2, State: Initialized, Role: FOLLOWER
I20260808 03:02:37.654198 31589 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ee56d9ac8e441b9c6b567be5da9be2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33897 } }
I20260808 03:02:37.654457 31589 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:37.654613 31589 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:37.654815 31589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:37.657248 31589 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ee56d9ac8e441b9c6b567be5da9be2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33897 } }
I20260808 03:02:37.657750 31589 leader_election.cc:302] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6ee56d9ac8e441b9c6b567be5da9be2; no voters:
I20260808 03:02:37.658404 31589 leader_election.cc:288] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:37.658540 31601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:37.659293 31601 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [term 1 LEADER]: Becoming Leader. State: Replica: b6ee56d9ac8e441b9c6b567be5da9be2, State: Running, Role: LEADER
I20260808 03:02:37.659893 31601 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ee56d9ac8e441b9c6b567be5da9be2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33897 } }
I20260808 03:02:37.660801 31589 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:37.663538 31602 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6ee56d9ac8e441b9c6b567be5da9be2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ee56d9ac8e441b9c6b567be5da9be2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33897 } } }
I20260808 03:02:37.663574 31605 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6ee56d9ac8e441b9c6b567be5da9be2. Latest consensus state: current_term: 1 leader_uuid: "b6ee56d9ac8e441b9c6b567be5da9be2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ee56d9ac8e441b9c6b567be5da9be2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33897 } } }
I20260808 03:02:37.664072 31602 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:37.664143 31605 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:37.666204 31618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:37.672251 31618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:37.682453 31618 catalog_manager.cc:1347] Generated new cluster ID: 982d527854e744f1ba9509714b2b436c
I20260808 03:02:37.682598 31618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:37.708822 31618 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:37.709359 31618 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:37.717972 31618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6ee56d9ac8e441b9c6b567be5da9be2: Generated new TSK 0
I20260808 03:02:37.718943 31618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:37.725692 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 31468
W20260808 03:02:37.727344 31577 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57121
W20260808 03:02:37.728058 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 31468: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:02:37.728194 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 31468
2026-08-08T03:02:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:37 dist-test-slave-6ntk krb5kdc[31649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:37 dist-test-slave-6ntk krb5kdc[31649](info): set up 2 sockets
Aug 08 03:02:37 dist-test-slave-6ntk krb5kdc[31649](info): commencing operation
krb5kdc: starting...
W20260808 03:02:39.097807 822 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.005s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:39 dist-test-slave-6ntk krb5kdc[31649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:39.449709 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:39.816267 31669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:39.816589 31669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:39.816685 31669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:39.824280 31669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:39.824409 31669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:39.824469 31669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:39.824525 31669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:39.824570 31669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:39.835111 31669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:39.836500 31669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:39.837797 31669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:39.850467 31674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:39.850360 31680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:39.850558 31677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:39.851809 31669 server_base.cc:1034] running on GCE node
I20260808 03:02:39.852695 31669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:39.854409 31669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:39.855669 31669 hybrid_clock.cc:648] HybridClock initialized: now 1786158159855626 us; error 57 us; skew 500 ppm
I20260808 03:02:39.856073 31669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:39 dist-test-slave-6ntk krb5kdc[31649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158159, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:39.859735 31669 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:39.861323 31669 webserver.cc:460] Webserver started at http://127.0.0.1:45729/ using document root <none> and password file <none>
I20260808 03:02:39.862157 31669 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:39.862318 31669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:39.862659 31669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:39.865835 31669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "84897ac67aec44a0ab2086bb5e2164c5"
format_stamp: "Formatted at 2026-08-08 03:02:39 on dist-test-slave-6ntk"
server_key: "a7bbcbd96bb0b508553f92c31e57814f"
server_key_iv: "8ab990066b07877013fc6d2e780eb35c"
server_key_version: "encryptionkey@0"
I20260808 03:02:39.866763 31669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "84897ac67aec44a0ab2086bb5e2164c5"
format_stamp: "Formatted at 2026-08-08 03:02:39 on dist-test-slave-6ntk"
server_key: "a7bbcbd96bb0b508553f92c31e57814f"
server_key_iv: "8ab990066b07877013fc6d2e780eb35c"
server_key_version: "encryptionkey@0"
I20260808 03:02:39.872296 31669 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:02:39.875960 31695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:39.877352 31669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:02:39.877575 31669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "84897ac67aec44a0ab2086bb5e2164c5"
format_stamp: "Formatted at 2026-08-08 03:02:39 on dist-test-slave-6ntk"
server_key: "a7bbcbd96bb0b508553f92c31e57814f"
server_key_iv: "8ab990066b07877013fc6d2e780eb35c"
server_key_version: "encryptionkey@0"
I20260808 03:02:39.877791 31669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:39.893761 31669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:39.898705 31669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:39.899024 31669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:39.918622 31669 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37695
I20260808 03:02:39.918643 31790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37695 every 8 connection(s)
I20260808 03:02:39.920150 31669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:39.924170 31791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:39.925009 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 31669
I20260808 03:02:39.925221 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:39.925791 822 external_mini_cluster.cc:1428] Setting key 8d91e1f3419a9f227f15b8e9347dab65
I20260808 03:02:39.939147 31791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5: Bootstrap starting.
Aug 08 03:02:39 dist-test-slave-6ntk krb5kdc[31649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 03:02:39.943233 31791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:39.944588 31791 log.cc:824] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:39.947731 31791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5: No bootstrap required, opened a new log
I20260808 03:02:39.951316 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:39.927735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52671 (local address 127.0.0.1:37695)
0808 03:02:39.928617 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:39.928629 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:39.928672 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:02:39.930894 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:39.930922 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:39.931391 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:39.931755 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:39.933671 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:39.935088 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:39.937089 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:39.937805 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:39.938038 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:39.943088 (+ 5050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:39.943097 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:39.943116 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:39.943157 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:39.946567 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:39.947334 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:39.947340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:39.947347 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:39.947524 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:39.947956 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:39.947962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:39.947965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:39.948745 (+ 780us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:39.949080 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:39.949381 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 03:02:39.949841 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":413,"threads_started":1}
I20260808 03:02:39.955250 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:39.957499 31791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84897ac67aec44a0ab2086bb5e2164c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37695 } }
I20260808 03:02:39.957978 31791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:39.958093 31791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84897ac67aec44a0ab2086bb5e2164c5, State: Initialized, Role: FOLLOWER
I20260808 03:02:39.958794 31791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84897ac67aec44a0ab2086bb5e2164c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37695 } }
I20260808 03:02:39.959139 31791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:39.959319 31791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:39.959530 31791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:39.962566 31791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84897ac67aec44a0ab2086bb5e2164c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37695 } }
I20260808 03:02:39.963104 31791 leader_election.cc:302] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84897ac67aec44a0ab2086bb5e2164c5; no voters:
I20260808 03:02:39.963807 31791 leader_election.cc:288] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:39.963946 31808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:39.965145 31808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [term 1 LEADER]: Becoming Leader. State: Replica: 84897ac67aec44a0ab2086bb5e2164c5, State: Running, Role: LEADER
I20260808 03:02:39.965777 31808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84897ac67aec44a0ab2086bb5e2164c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37695 } }
I20260808 03:02:39.966130 31791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:39.971700 31813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84897ac67aec44a0ab2086bb5e2164c5. Latest consensus state: current_term: 1 leader_uuid: "84897ac67aec44a0ab2086bb5e2164c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84897ac67aec44a0ab2086bb5e2164c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37695 } } }
I20260808 03:02:39.972100 31813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:39.975804 31809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84897ac67aec44a0ab2086bb5e2164c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84897ac67aec44a0ab2086bb5e2164c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37695 } } }
I20260808 03:02:39.976178 31809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:39.979043 31821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:39.980985 31794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:39.960757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53059 (local address 127.0.0.1:37695)
0808 03:02:39.960963 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:39.960973 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:39.961468 (+ 495us) server_negotiation.cc:407] Connection header received
0808 03:02:39.961616 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:39.961623 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:39.961712 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:39.961814 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:39.963464 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:39.964701 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:39.966678 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:39.967637 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:39.967805 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:39.972381 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:39.972395 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:39.972403 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:39.972456 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:39.977165 (+ 4709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:39.978233 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:39.978242 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:39.978252 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:39.978467 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:39.979292 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:39.979299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:39.979304 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:39.979744 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:39.980024 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:39.980367 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 03:02:39.980596 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 03:02:39.984622 31821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:39.994165 31821 catalog_manager.cc:1347] Generated new cluster ID: 293eb89b8a5944b4a7c86e6edee59a4e
I20260808 03:02:39.994302 31821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:40.014993 31821 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:40.016412 31821 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:40.029176 31821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84897ac67aec44a0ab2086bb5e2164c5: Generated new TSK 0
I20260808 03:02:40.030470 31821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:40.045768 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 31669
I20260808 03:02:40.047662 31778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:40.143728 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 31669
2026-08-08T03:02:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:40 dist-test-slave-6ntk krb5kdc[31847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:40 dist-test-slave-6ntk krb5kdc[31847](info): set up 2 sockets
Aug 08 03:02:40 dist-test-slave-6ntk krb5kdc[31847](info): commencing operation
krb5kdc: starting...
W20260808 03:02:41.487344 822 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.003s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:41 dist-test-slave-6ntk krb5kdc[31847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:41.889459 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:42.246805 31867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:42.247123 31867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:42.247219 31867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:42.254626 31867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:42.254751 31867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:42.254799 31867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:42.254863 31867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:42.254900 31867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:42.265113 31867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:42.266628 31867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:42.267964 31867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:42.280275 31874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:42.280373 31875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:42.280330 31877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:42.281522 31867 server_base.cc:1034] running on GCE node
I20260808 03:02:42.282418 31867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:42.283872 31867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:42.285090 31867 hybrid_clock.cc:648] HybridClock initialized: now 1786158162285050 us; error 30 us; skew 500 ppm
I20260808 03:02:42.285436 31867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:42 dist-test-slave-6ntk krb5kdc[31847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158162, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:42.288611 31867 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:42.290244 31867 webserver.cc:460] Webserver started at http://127.0.0.1:36383/ using document root <none> and password file <none>
I20260808 03:02:42.290959 31867 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:42.291112 31867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:42.291441 31867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:42.294629 31867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "4d5e3b4d39ca45c2adf41f911e78e411"
format_stamp: "Formatted at 2026-08-08 03:02:42 on dist-test-slave-6ntk"
server_key: "b3b7466306468803776dacc852933fd8"
server_key_iv: "be6b77ab4ac6280a85b0b847f0f0a85c"
server_key_version: "encryptionkey@0"
I20260808 03:02:42.295428 31867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "4d5e3b4d39ca45c2adf41f911e78e411"
format_stamp: "Formatted at 2026-08-08 03:02:42 on dist-test-slave-6ntk"
server_key: "b3b7466306468803776dacc852933fd8"
server_key_iv: "be6b77ab4ac6280a85b0b847f0f0a85c"
server_key_version: "encryptionkey@0"
I20260808 03:02:42.300681 31867 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:02:42.304301 31889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:42.305650 31867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:02:42.305848 31867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "4d5e3b4d39ca45c2adf41f911e78e411"
format_stamp: "Formatted at 2026-08-08 03:02:42 on dist-test-slave-6ntk"
server_key: "b3b7466306468803776dacc852933fd8"
server_key_iv: "be6b77ab4ac6280a85b0b847f0f0a85c"
server_key_version: "encryptionkey@0"
I20260808 03:02:42.306136 31867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:42.329051 31867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:42.333460 31867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:42.333782 31867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:42.355782 31867 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44443
I20260808 03:02:42.355799 31971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44443 every 8 connection(s)
I20260808 03:02:42.357347 31867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:42.361267 31972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:42.364025 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 31867
I20260808 03:02:42.364226 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:42.364769 822 external_mini_cluster.cc:1428] Setting key 999d6c492c6ca2295d4786e278b915f2
I20260808 03:02:42.374001 31972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411: Bootstrap starting.
I20260808 03:02:42.377835 31972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:42.379078 31972 log.cc:824] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:42.379235 31978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:42.366851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57253 (local address 127.0.0.1:44443)
0808 03:02:42.367794 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:42.367806 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:42.367852 (+ 46us) server_negotiation.cc:407] Connection header received
0808 03:02:42.369713 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:42.369745 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:42.370267 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:42.370644 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:42.371258 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:42.372779 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:42.374746 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:42.375418 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:42.375567 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:42.376882 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:42.376890 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:42.376903 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:42.377437 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:42.377510 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:42.377535 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:42.377627 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:42.377983 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 03:02:42.378228 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":462,"threads_started":1}
I20260808 03:02:42.382874 31972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411: No bootstrap required, opened a new log
I20260808 03:02:42.384895 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:42.391534 31972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5e3b4d39ca45c2adf41f911e78e411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44443 } }
I20260808 03:02:42.392032 31972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:42.392151 31972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d5e3b4d39ca45c2adf41f911e78e411, State: Initialized, Role: FOLLOWER
I20260808 03:02:42.392822 31972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5e3b4d39ca45c2adf41f911e78e411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44443 } }
I20260808 03:02:42.393061 31972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:42.393225 31972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:42.393405 31972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:42.396556 31978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:42.389586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57117 (local address 127.0.0.1:44443)
0808 03:02:42.389784 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:42.389790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:42.390110 (+ 320us) server_negotiation.cc:407] Connection header received
0808 03:02:42.390306 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:42.390311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:42.390395 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:42.390510 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:42.391045 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:42.392145 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:42.393746 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:42.394495 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:42.394638 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:42.395786 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:42.395792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:42.395797 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:42.395819 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:42.395865 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:42.395884 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:42.395976 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:42.396074 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 03:02:42.396171 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:02:42.396479 31972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5e3b4d39ca45c2adf41f911e78e411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44443 } }
I20260808 03:02:42.397107 31972 leader_election.cc:302] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d5e3b4d39ca45c2adf41f911e78e411; no voters:
I20260808 03:02:42.397791 31972 leader_election.cc:288] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:42.397929 31990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:42.399755 31972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:42.401355 31990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [term 1 LEADER]: Becoming Leader. State: Replica: 4d5e3b4d39ca45c2adf41f911e78e411, State: Running, Role: LEADER
I20260808 03:02:42.402026 31990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5e3b4d39ca45c2adf41f911e78e411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44443 } }
I20260808 03:02:42.405493 31992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d5e3b4d39ca45c2adf41f911e78e411. Latest consensus state: current_term: 1 leader_uuid: "4d5e3b4d39ca45c2adf41f911e78e411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5e3b4d39ca45c2adf41f911e78e411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44443 } } }
I20260808 03:02:42.405576 31991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d5e3b4d39ca45c2adf41f911e78e411" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5e3b4d39ca45c2adf41f911e78e411" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44443 } } }
I20260808 03:02:42.405989 31992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:42.406061 31991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:42.418243 32000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:42.424108 32000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:42.432909 32000 catalog_manager.cc:1347] Generated new cluster ID: af0092e63f034874b311431fb0bcd130
I20260808 03:02:42.433041 32000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:42.455082 32000 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:42.457166 32000 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:42.472867 32000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d5e3b4d39ca45c2adf41f911e78e411: Generated new TSK 0
I20260808 03:02:42.474139 32000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:42.490368 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 31867
W20260808 03:02:42.491603 31964 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57253
W20260808 03:02:42.492372 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 31867: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:02:42.492547 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 31867
2026-08-08T03:02:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:42 dist-test-slave-6ntk krb5kdc[32028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:42 dist-test-slave-6ntk krb5kdc[32028](info): set up 2 sockets
Aug 08 03:02:42 dist-test-slave-6ntk krb5kdc[32028](info): commencing operation
krb5kdc: starting...
W20260808 03:02:43.935037 822 mini_kdc.cc:121] Time spent starting KDC: real 1.397s user 0.005s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:44 dist-test-slave-6ntk krb5kdc[32028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:44.281294 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:44.648645 32048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:44.649046 32048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:44.649224 32048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:44.656879 32048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:44.657047 32048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:44.657152 32048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:44.657274 32048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:44.657353 32048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:44.667518 32048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:44.668951 32048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:44.670382 32048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:44.682998 32056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:44.682976 32057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:44.683234 32059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:44.683956 32048 server_base.cc:1034] running on GCE node
I20260808 03:02:44.684826 32048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:44.686379 32048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:44.687619 32048 hybrid_clock.cc:648] HybridClock initialized: now 1786158164687563 us; error 54 us; skew 500 ppm
I20260808 03:02:44.687984 32048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:44 dist-test-slave-6ntk krb5kdc[32028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158164, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:44.691450 32048 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:44.692968 32048 webserver.cc:460] Webserver started at http://127.0.0.1:43393/ using document root <none> and password file <none>
I20260808 03:02:44.693756 32048 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:44.693928 32048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:44.694263 32048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:44.697294 32048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "daf56a46b7a5451eafcd63d5789c1e17"
format_stamp: "Formatted at 2026-08-08 03:02:44 on dist-test-slave-6ntk"
server_key: "63f9c1a9b6a96a3bb66c1055a31fe280"
server_key_iv: "0dfa23019369cefcbc268280c4ec69da"
server_key_version: "encryptionkey@0"
I20260808 03:02:44.698148 32048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "daf56a46b7a5451eafcd63d5789c1e17"
format_stamp: "Formatted at 2026-08-08 03:02:44 on dist-test-slave-6ntk"
server_key: "63f9c1a9b6a96a3bb66c1055a31fe280"
server_key_iv: "0dfa23019369cefcbc268280c4ec69da"
server_key_version: "encryptionkey@0"
I20260808 03:02:44.703505 32048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 03:02:44.707017 32068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:44.708336 32048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 03:02:44.708542 32048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "daf56a46b7a5451eafcd63d5789c1e17"
format_stamp: "Formatted at 2026-08-08 03:02:44 on dist-test-slave-6ntk"
server_key: "63f9c1a9b6a96a3bb66c1055a31fe280"
server_key_iv: "0dfa23019369cefcbc268280c4ec69da"
server_key_version: "encryptionkey@0"
I20260808 03:02:44.708765 32048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:44.714654 32048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:44.719209 32048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:44.719532 32048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:44.739512 32048 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36751
I20260808 03:02:44.739605 32145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36751 every 8 connection(s)
I20260808 03:02:44.740980 32048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:44.744855 32146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:44.744845 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 32048
I20260808 03:02:44.745065 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:44.745533 822 external_mini_cluster.cc:1428] Setting key 49d3eb839c8340119c463a7f8935c8aa
I20260808 03:02:44.753577 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:44.747498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59279 (local address 127.0.0.1:36751)
0808 03:02:44.748431 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:44.748445 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:44.748488 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:02:44.750073 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:44.750095 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:44.750436 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:44.750770 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:44.751973 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:44.751980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:44.751992 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:44.752006 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:44.752073 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:44.752096 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:44.752286 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:44.752394 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 03:02:44.752613 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":436,"threads_started":1}
I20260808 03:02:44.757539 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:44.759089 32146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17: Bootstrap starting.
I20260808 03:02:44.762612 32146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:44.763782 32146 log.cc:824] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:44.765075 32151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:44.762220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57143 (local address 127.0.0.1:36751)
0808 03:02:44.762448 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:44.762458 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:44.763011 (+ 553us) server_negotiation.cc:407] Connection header received
0808 03:02:44.763217 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:44.763226 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:44.763278 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:44.763390 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:44.764359 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:44.764374 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:44.764379 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:44.764402 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:44.764442 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:44.764457 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:44.764597 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:44.764725 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 03:02:44.764790 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 03:02:44.767197 32146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17: No bootstrap required, opened a new log
I20260808 03:02:44.775836 32146 raft_consensus.cc:348] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf56a46b7a5451eafcd63d5789c1e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } }
I20260808 03:02:44.776389 32146 raft_consensus.cc:374] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:44.776510 32146 raft_consensus.cc:729] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daf56a46b7a5451eafcd63d5789c1e17, State: Initialized, Role: FOLLOWER
I20260808 03:02:44.777100 32146 consensus_queue.cc:260] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf56a46b7a5451eafcd63d5789c1e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } }
I20260808 03:02:44.777354 32146 raft_consensus.cc:388] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:44.777468 32146 raft_consensus.cc:482] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:44.777611 32146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:44.780210 32146 raft_consensus.cc:504] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf56a46b7a5451eafcd63d5789c1e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } }
I20260808 03:02:44.780807 32146 leader_election.cc:302] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daf56a46b7a5451eafcd63d5789c1e17; no voters:
I20260808 03:02:44.781643 32146 leader_election.cc:288] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:44.781795 32160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:44.782622 32160 raft_consensus.cc:686] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [term 1 LEADER]: Becoming Leader. State: Replica: daf56a46b7a5451eafcd63d5789c1e17, State: Running, Role: LEADER
I20260808 03:02:44.783196 32160 consensus_queue.cc:237] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf56a46b7a5451eafcd63d5789c1e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } }
I20260808 03:02:44.784101 32146 sys_catalog.cc:564] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:44.786532 32165 sys_catalog.cc:455] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daf56a46b7a5451eafcd63d5789c1e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf56a46b7a5451eafcd63d5789c1e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } } }
I20260808 03:02:44.787097 32165 sys_catalog.cc:458] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:44.787822 32168 sys_catalog.cc:455] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader daf56a46b7a5451eafcd63d5789c1e17. Latest consensus state: current_term: 1 leader_uuid: "daf56a46b7a5451eafcd63d5789c1e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daf56a46b7a5451eafcd63d5789c1e17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36751 } } }
I20260808 03:02:44.788210 32168 sys_catalog.cc:458] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:44.789924 32180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:44.795835 32180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:44.808988 32180 catalog_manager.cc:1347] Generated new cluster ID: 2d144bf3e1824d8a9f2fb1251d650f3c
I20260808 03:02:44.809165 32180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:44.846967 32180 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:44.847496 32180 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:44.867590 32180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P daf56a46b7a5451eafcd63d5789c1e17: Generated new TSK 0
I20260808 03:02:44.868571 32180 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:44.876693 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 32048
W20260808 03:02:44.877974 32141 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59279
W20260808 03:02:44.878698 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 32048: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:02:44.878849 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 32048
2026-08-08T03:02:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:45 dist-test-slave-6ntk krb5kdc[32197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:45 dist-test-slave-6ntk krb5kdc[32197](info): set up 2 sockets
Aug 08 03:02:45 dist-test-slave-6ntk krb5kdc[32197](info): commencing operation
krb5kdc: starting...
W20260808 03:02:46.242341 822 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:46 dist-test-slave-6ntk krb5kdc[32197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:46.589506 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34499
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:46.944696 32214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:46.945082 32214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:46.945261 32214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:46.952605 32214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:46.952785 32214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:46.952901 32214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:46.953016 32214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:46.953104 32214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:46.963315 32214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:46.964717 32214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:46.966126 32214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:46.978771 32223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:46.978868 32228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:46.978876 32222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:46.979907 32214 server_base.cc:1034] running on GCE node
I20260808 03:02:46.980840 32214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:46.982681 32214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:46.983903 32214 hybrid_clock.cc:648] HybridClock initialized: now 1786158166983850 us; error 46 us; skew 500 ppm
I20260808 03:02:46.984236 32214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:46 dist-test-slave-6ntk krb5kdc[32197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158166, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:46.992041 32214 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:46.993779 32214 webserver.cc:460] Webserver started at http://127.0.0.1:43203/ using document root <none> and password file <none>
I20260808 03:02:46.994575 32214 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:46.994719 32214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:46.995055 32214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:46.998318 32214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "be0fddf0bd614a3dbb28d78021ae15ac"
format_stamp: "Formatted at 2026-08-08 03:02:46 on dist-test-slave-6ntk"
server_key: "76da590936975dce83382d78d6fb0a39"
server_key_iv: "92c9cca159785974d7bdd66ea1eef55a"
server_key_version: "encryptionkey@0"
I20260808 03:02:46.999145 32214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "be0fddf0bd614a3dbb28d78021ae15ac"
format_stamp: "Formatted at 2026-08-08 03:02:46 on dist-test-slave-6ntk"
server_key: "76da590936975dce83382d78d6fb0a39"
server_key_iv: "92c9cca159785974d7bdd66ea1eef55a"
server_key_version: "encryptionkey@0"
I20260808 03:02:47.004743 32214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:02:47.008486 32240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:47.009958 32214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 03:02:47.010155 32214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "be0fddf0bd614a3dbb28d78021ae15ac"
format_stamp: "Formatted at 2026-08-08 03:02:46 on dist-test-slave-6ntk"
server_key: "76da590936975dce83382d78d6fb0a39"
server_key_iv: "92c9cca159785974d7bdd66ea1eef55a"
server_key_version: "encryptionkey@0"
I20260808 03:02:47.010390 32214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:47.024741 32214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:47.029556 32214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:47.030020 32214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:47.051424 32214 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34499
I20260808 03:02:47.051430 32315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34499 every 8 connection(s)
I20260808 03:02:47.053015 32214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:47.054246 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 32214
I20260808 03:02:47.054463 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:47.055054 822 external_mini_cluster.cc:1428] Setting key 5cf073231cbd77e4a9120752fcd12013
I20260808 03:02:47.058498 32316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 03:02:47 dist-test-slave-6ntk krb5kdc[32197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 03:02:47.069623 32316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac: Bootstrap starting.
I20260808 03:02:47.073233 32316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:47.074401 32316 log.cc:824] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:47.077744 32316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac: No bootstrap required, opened a new log
I20260808 03:02:47.077831 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:47.056962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:59417 (local address 127.0.0.1:34499)
0808 03:02:47.057740 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:47.057753 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:47.057796 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:02:47.059614 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:47.059643 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:47.060028 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:47.060438 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:47.062182 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:47.063544 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:47.064828 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:47.065513 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:47.065711 (+ 198us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:47.070012 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:47.070021 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:47.070036 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:47.070079 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:47.073191 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:47.074040 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:47.074047 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:47.074053 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:47.074227 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:47.074753 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:47.074757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:47.074759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:47.075585 (+ 826us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:47.075923 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:47.076474 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 03:02:47.076821 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":355,"threads_started":1}
I20260808 03:02:47.082122 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:47.086719 32316 raft_consensus.cc:348] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be0fddf0bd614a3dbb28d78021ae15ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34499 } }
I20260808 03:02:47.087242 32316 raft_consensus.cc:374] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:47.087387 32316 raft_consensus.cc:729] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be0fddf0bd614a3dbb28d78021ae15ac, State: Initialized, Role: FOLLOWER
I20260808 03:02:47.088050 32316 consensus_queue.cc:260] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be0fddf0bd614a3dbb28d78021ae15ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34499 } }
I20260808 03:02:47.088302 32316 raft_consensus.cc:388] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:47.088474 32316 raft_consensus.cc:482] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:47.088671 32316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:47.091095 32316 raft_consensus.cc:504] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be0fddf0bd614a3dbb28d78021ae15ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34499 } }
I20260808 03:02:47.091636 32316 leader_election.cc:302] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be0fddf0bd614a3dbb28d78021ae15ac; no voters:
I20260808 03:02:47.092361 32316 leader_election.cc:288] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:47.092505 32330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:47.094146 32330 raft_consensus.cc:686] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [term 1 LEADER]: Becoming Leader. State: Replica: be0fddf0bd614a3dbb28d78021ae15ac, State: Running, Role: LEADER
I20260808 03:02:47.094833 32330 consensus_queue.cc:237] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be0fddf0bd614a3dbb28d78021ae15ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34499 } }
I20260808 03:02:47.095117 32316 sys_catalog.cc:564] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:47.102849 32331 sys_catalog.cc:455] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be0fddf0bd614a3dbb28d78021ae15ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be0fddf0bd614a3dbb28d78021ae15ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34499 } } }
I20260808 03:02:47.103278 32331 sys_catalog.cc:458] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:47.103842 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:47.086843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:57449 (local address 127.0.0.1:34499)
0808 03:02:47.087051 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:47.087057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:47.087505 (+ 448us) server_negotiation.cc:407] Connection header received
0808 03:02:47.087706 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:47.087711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:47.087788 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:47.087903 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:47.089561 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:47.090477 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:47.091849 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:47.092364 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:47.092512 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:47.095598 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:47.095603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:47.095606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:47.095641 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:47.099076 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:47.099942 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:47.099947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:47.099950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:47.100104 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:47.100590 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:47.100595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:47.100598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:47.100990 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:47.102666 (+ 1676us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:47.102922 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 03:02:47.103202 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:02:47.103737 32332 sys_catalog.cc:455] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader be0fddf0bd614a3dbb28d78021ae15ac. Latest consensus state: current_term: 1 leader_uuid: "be0fddf0bd614a3dbb28d78021ae15ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be0fddf0bd614a3dbb28d78021ae15ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34499 } } }
I20260808 03:02:47.104252 32332 sys_catalog.cc:458] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:47.106691 32349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:47.111960 32349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:47.121461 32349 catalog_manager.cc:1347] Generated new cluster ID: c9873a3154a94b42886d9f804b802618
I20260808 03:02:47.121599 32349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:47.137599 32349 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:47.139078 32349 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:47.149662 32349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be0fddf0bd614a3dbb28d78021ae15ac: Generated new TSK 0
I20260808 03:02:47.150893 32349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:47.161679 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 32214
I20260808 03:02:47.162838 32309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:47.257751 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 32214
W20260808 03:02:47.280930 32194 connection.cc:535] client connection to 127.0.0.1:34499 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34499): Connection reset by peer (error 104)
2026-08-08T03:02:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2377 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:47 dist-test-slave-6ntk krb5kdc[32365](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:47 dist-test-slave-6ntk krb5kdc[32365](info): set up 2 sockets
Aug 08 03:02:47 dist-test-slave-6ntk krb5kdc[32365](info): commencing operation
krb5kdc: starting...
W20260808 03:02:48.648752 822 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:48 dist-test-slave-6ntk krb5kdc[32365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:48.997442 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:49.363003 32384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:49.363309 32384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:49.363502 32384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:49.371033 32384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:49.371148 32384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:49.371264 32384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:49.371376 32384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:49.371441 32384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:49.381466 32384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:49.382952 32384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:49.384284 32384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:49.396484 32394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:49.396713 32396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:49.397135 32398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:49.397549 32384 server_base.cc:1034] running on GCE node
I20260808 03:02:49.398471 32384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:49.400012 32384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:49.401222 32384 hybrid_clock.cc:648] HybridClock initialized: now 1786158169401148 us; error 72 us; skew 500 ppm
I20260808 03:02:49.401607 32384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:49 dist-test-slave-6ntk krb5kdc[32365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:49.405141 32384 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:49.406844 32384 webserver.cc:460] Webserver started at http://127.0.0.1:37569/ using document root <none> and password file <none>
I20260808 03:02:49.407625 32384 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:49.407774 32384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:49.408116 32384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:49.411846 32384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "28993bd184c44f95a84579d2b5b9ad1d"
format_stamp: "Formatted at 2026-08-08 03:02:49 on dist-test-slave-6ntk"
server_key: "3e6e66b38a3c9dc56afc36cfc241b710"
server_key_iv: "9d3511ae371c9a6a8440ccafe03da684"
server_key_version: "encryptionkey@0"
I20260808 03:02:49.412981 32384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "28993bd184c44f95a84579d2b5b9ad1d"
format_stamp: "Formatted at 2026-08-08 03:02:49 on dist-test-slave-6ntk"
server_key: "3e6e66b38a3c9dc56afc36cfc241b710"
server_key_iv: "9d3511ae371c9a6a8440ccafe03da684"
server_key_version: "encryptionkey@0"
I20260808 03:02:49.418892 32384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 03:02:49.422866 32409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:49.424319 32384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:49.424531 32384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "28993bd184c44f95a84579d2b5b9ad1d"
format_stamp: "Formatted at 2026-08-08 03:02:49 on dist-test-slave-6ntk"
server_key: "3e6e66b38a3c9dc56afc36cfc241b710"
server_key_iv: "9d3511ae371c9a6a8440ccafe03da684"
server_key_version: "encryptionkey@0"
I20260808 03:02:49.424758 32384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:49.466797 32384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:49.471405 32384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:49.471710 32384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:49.491043 32384 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44937
I20260808 03:02:49.491065 32492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44937 every 8 connection(s)
I20260808 03:02:49.492588 32384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:49.494671 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 32384
I20260808 03:02:49.494905 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:49.495405 822 external_mini_cluster.cc:1428] Setting key 14444c99a016b7ef40d61ce5e86b9d3a
I20260808 03:02:49.497159 32496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:49.511197 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:49.497476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:45901 (local address 127.0.0.1:44937)
0808 03:02:49.498281 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:49.498298 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:49.498351 (+ 53us) server_negotiation.cc:407] Connection header received
0808 03:02:49.500506 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:49.500537 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:49.501087 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:49.501562 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:49.502120 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:49.504207 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:49.505577 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:49.506688 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:49.506879 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:49.508111 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:49.508123 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:49.508152 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:49.508971 (+ 819us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:49.509090 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:49.509127 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:49.509301 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:49.509583 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 03:02:49.509785 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":351,"threads_started":1}
I20260808 03:02:49.512812 32496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d: Bootstrap starting.
I20260808 03:02:49.515627 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:49.516919 32496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:49.518306 32496 log.cc:824] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:49.521759 32496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d: No bootstrap required, opened a new log
I20260808 03:02:49.527915 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:49.521179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:44783 (local address 127.0.0.1:44937)
0808 03:02:49.521343 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:49.521348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:49.521904 (+ 556us) server_negotiation.cc:407] Connection header received
0808 03:02:49.522157 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:49.522162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:49.522229 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:49.522356 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:49.522965 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:49.523773 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:49.525164 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:49.525632 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:49.525811 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:49.527084 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:49.527089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:49.527092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:49.527111 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:49.527172 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:49.527187 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:49.527327 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:49.527554 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 03:02:49.527617 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:02:49.530920 32496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28993bd184c44f95a84579d2b5b9ad1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44937 } }
I20260808 03:02:49.531358 32496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:49.531477 32496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28993bd184c44f95a84579d2b5b9ad1d, State: Initialized, Role: FOLLOWER
I20260808 03:02:49.532099 32496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28993bd184c44f95a84579d2b5b9ad1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44937 } }
I20260808 03:02:49.532369 32496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:49.532528 32496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:49.532727 32496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:49.535478 32496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28993bd184c44f95a84579d2b5b9ad1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44937 } }
I20260808 03:02:49.536059 32496 leader_election.cc:302] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28993bd184c44f95a84579d2b5b9ad1d; no voters:
I20260808 03:02:49.536724 32496 leader_election.cc:288] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:49.537010 32516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:49.538232 32516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [term 1 LEADER]: Becoming Leader. State: Replica: 28993bd184c44f95a84579d2b5b9ad1d, State: Running, Role: LEADER
I20260808 03:02:49.538863 32516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28993bd184c44f95a84579d2b5b9ad1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44937 } }
I20260808 03:02:49.539392 32496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:49.542297 32518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28993bd184c44f95a84579d2b5b9ad1d. Latest consensus state: current_term: 1 leader_uuid: "28993bd184c44f95a84579d2b5b9ad1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28993bd184c44f95a84579d2b5b9ad1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44937 } } }
I20260808 03:02:49.542364 32517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28993bd184c44f95a84579d2b5b9ad1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28993bd184c44f95a84579d2b5b9ad1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44937 } } }
I20260808 03:02:49.542940 32518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:49.543002 32517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:49.544250 32532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:49.552453 32532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:49.562793 32532 catalog_manager.cc:1347] Generated new cluster ID: 891024ad6d384cdd9a25e91d8ce9b8cf
I20260808 03:02:49.563006 32532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:49.577673 32532 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:49.579831 32532 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:49.593600 32532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28993bd184c44f95a84579d2b5b9ad1d: Generated new TSK 0
I20260808 03:02:49.594584 32532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:49.601732 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 32384
W20260808 03:02:49.603207 32485 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.193:45901
W20260808 03:02:49.604166 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 32384: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:02:49.604380 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 32384
W20260808 03:02:49.634512 32363 connection.cc:535] client connection to 127.0.0.1:44937 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44937): Connection reset by peer (error 104)
2026-08-08T03:02:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2353 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:49 dist-test-slave-6ntk krb5kdc[32551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:49 dist-test-slave-6ntk krb5kdc[32551](info): set up 2 sockets
Aug 08 03:02:49 dist-test-slave-6ntk krb5kdc[32551](info): commencing operation
krb5kdc: starting...
W20260808 03:02:50.978148 822 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.008s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:51 dist-test-slave-6ntk krb5kdc[32551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:51.334015 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:51.696995 32572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:51.697314 32572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:51.697412 32572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:51.705411 32572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:51.705533 32572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:51.705588 32572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:51.705653 32572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:51.705698 32572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:51.715909 32572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:51.717232 32572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:51.718611 32572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:51.731451 32578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:51.731361 32591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:51.731405 32581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:51.732354 32572 server_base.cc:1034] running on GCE node
I20260808 03:02:51.733209 32572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:51.734746 32572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:51.735978 32572 hybrid_clock.cc:648] HybridClock initialized: now 1786158171735911 us; error 55 us; skew 500 ppm
I20260808 03:02:51.736380 32572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:51 dist-test-slave-6ntk krb5kdc[32551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158171, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:51.739856 32572 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:51.741494 32572 webserver.cc:460] Webserver started at http://127.0.0.1:33541/ using document root <none> and password file <none>
I20260808 03:02:51.742285 32572 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:51.742396 32572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:51.742672 32572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:51.745766 32572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "74c04483132748d3af84b5ead8dcffb6"
format_stamp: "Formatted at 2026-08-08 03:02:51 on dist-test-slave-6ntk"
server_key: "52db265cb273a267bb8fc057f17a7bb1"
server_key_iv: "886b04ed1dd51dafad1cba0e4f9ef59d"
server_key_version: "encryptionkey@0"
I20260808 03:02:51.746651 32572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "74c04483132748d3af84b5ead8dcffb6"
format_stamp: "Formatted at 2026-08-08 03:02:51 on dist-test-slave-6ntk"
server_key: "52db265cb273a267bb8fc057f17a7bb1"
server_key_iv: "886b04ed1dd51dafad1cba0e4f9ef59d"
server_key_version: "encryptionkey@0"
I20260808 03:02:51.752393 32572 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:02:51.756081 32601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:51.757488 32572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:02:51.757702 32572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "74c04483132748d3af84b5ead8dcffb6"
format_stamp: "Formatted at 2026-08-08 03:02:51 on dist-test-slave-6ntk"
server_key: "52db265cb273a267bb8fc057f17a7bb1"
server_key_iv: "886b04ed1dd51dafad1cba0e4f9ef59d"
server_key_version: "encryptionkey@0"
I20260808 03:02:51.757992 32572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:51.764045 32572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:51.768404 32572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:51.768738 32572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:51.788303 32572 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38607
I20260808 03:02:51.788342 32682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38607 every 8 connection(s)
I20260808 03:02:51.789852 32572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:51.793979 32683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:51.798297 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 32572
I20260808 03:02:51.798520 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:51.798977 822 external_mini_cluster.cc:1428] Setting key 78f10c769859884d91a5ea7ddb50519b
I20260808 03:02:51.809002 32690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:51.800938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:59857 (local address 127.0.0.1:38607)
0808 03:02:51.801829 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:51.801846 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:51.801923 (+ 77us) server_negotiation.cc:407] Connection header received
0808 03:02:51.804458 (+ 2535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:51.804498 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:51.805046 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:51.805568 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:51.806787 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:51.806796 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:51.806816 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:51.806844 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:51.806926 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:51.806960 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:51.807219 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:51.807389 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 03:02:51.807617 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":351,"threads_started":1}
I20260808 03:02:51.809844 32683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6: Bootstrap starting.
I20260808 03:02:51.813580 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:51.814431 32683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:51.815858 32683 log.cc:824] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:51.819965 32683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6: No bootstrap required, opened a new log
I20260808 03:02:51.822281 32690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:51.819470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:44825 (local address 127.0.0.1:38607)
0808 03:02:51.819683 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:51.819689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:51.820223 (+ 534us) server_negotiation.cc:407] Connection header received
0808 03:02:51.820354 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:51.820359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:51.820422 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:51.820573 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:51.821489 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:51.821495 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:51.821500 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:51.821522 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:51.821574 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:51.821598 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:51.821789 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:51.821841 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 03:02:51.821944 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 03:02:51.829500 32683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c04483132748d3af84b5ead8dcffb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260808 03:02:51.829999 32683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:51.830099 32683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c04483132748d3af84b5ead8dcffb6, State: Initialized, Role: FOLLOWER
I20260808 03:02:51.830663 32683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c04483132748d3af84b5ead8dcffb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260808 03:02:51.830890 32683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:51.831001 32683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:51.831147 32683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:51.833755 32683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c04483132748d3af84b5ead8dcffb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260808 03:02:51.834326 32683 leader_election.cc:302] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74c04483132748d3af84b5ead8dcffb6; no voters:
I20260808 03:02:51.834939 32683 leader_election.cc:288] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:51.835094 32697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:51.835983 32697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [term 1 LEADER]: Becoming Leader. State: Replica: 74c04483132748d3af84b5ead8dcffb6, State: Running, Role: LEADER
I20260808 03:02:51.836560 32697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c04483132748d3af84b5ead8dcffb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260808 03:02:51.837457 32683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:51.839785 32698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74c04483132748d3af84b5ead8dcffb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c04483132748d3af84b5ead8dcffb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } } }
I20260808 03:02:51.840265 32698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:51.845795 32699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74c04483132748d3af84b5ead8dcffb6. Latest consensus state: current_term: 1 leader_uuid: "74c04483132748d3af84b5ead8dcffb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74c04483132748d3af84b5ead8dcffb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } } }
I20260808 03:02:51.846783 32699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:51.847095 32707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:51.853181 32707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:51.863713 32707 catalog_manager.cc:1347] Generated new cluster ID: 04d56fa96d5a438bac3c5b46382eea11
I20260808 03:02:51.863891 32707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:51.883078 32707 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:51.883517 32707 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:51.891985 32707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74c04483132748d3af84b5ead8dcffb6: Generated new TSK 0
I20260808 03:02:51.893054 32707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:51.910038 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 32572
W20260808 03:02:51.911517 32678 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.193:59857
W20260808 03:02:51.912340 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 32572: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:02:51.912519 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 32572
2026-08-08T03:02:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:52 dist-test-slave-6ntk krb5kdc[32729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:52 dist-test-slave-6ntk krb5kdc[32729](info): set up 2 sockets
Aug 08 03:02:52 dist-test-slave-6ntk krb5kdc[32729](info): commencing operation
krb5kdc: starting...
W20260808 03:02:53.309320 822 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:53 dist-test-slave-6ntk krb5kdc[32729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:53.681296 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40621
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:54.046862 32747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:54.047284 32747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:54.047464 32747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:54.054991 32747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:54.055182 32747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:54.055306 32747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:54.055425 32747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:54.055519 32747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:54.066143 32747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40621
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:54.067639 32747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:54.069061 32747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:54.081676 32756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:54.082217 32760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:54.081704 32754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:54.084069 32747 server_base.cc:1034] running on GCE node
I20260808 03:02:54.085155 32747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:54.087018 32747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:54.088418 32747 hybrid_clock.cc:648] HybridClock initialized: now 1786158174088324 us; error 60 us; skew 500 ppm
I20260808 03:02:54.088855 32747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:54 dist-test-slave-6ntk krb5kdc[32729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:54.092231 32747 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:54.094009 32747 webserver.cc:460] Webserver started at http://127.0.0.1:45639/ using document root <none> and password file <none>
I20260808 03:02:54.094777 32747 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:54.094940 32747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:54.095376 32747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:54.098658 32747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "7a72575dc16841b8a1b5659f85f4d917"
format_stamp: "Formatted at 2026-08-08 03:02:54 on dist-test-slave-6ntk"
server_key: "bf7cc2221d1e8eab56ba42c401a24a27"
server_key_iv: "7e2b9860ef70964211523d21e802db40"
server_key_version: "encryptionkey@0"
I20260808 03:02:54.099429 32747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "7a72575dc16841b8a1b5659f85f4d917"
format_stamp: "Formatted at 2026-08-08 03:02:54 on dist-test-slave-6ntk"
server_key: "bf7cc2221d1e8eab56ba42c401a24a27"
server_key_iv: "7e2b9860ef70964211523d21e802db40"
server_key_version: "encryptionkey@0"
I20260808 03:02:54.104851 32747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:02:54.108814 302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:54.110419 32747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 03:02:54.110574 32747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "7a72575dc16841b8a1b5659f85f4d917"
format_stamp: "Formatted at 2026-08-08 03:02:54 on dist-test-slave-6ntk"
server_key: "bf7cc2221d1e8eab56ba42c401a24a27"
server_key_iv: "7e2b9860ef70964211523d21e802db40"
server_key_version: "encryptionkey@0"
I20260808 03:02:54.110754 32747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:54.130755 32747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:54.135008 32747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:54.135257 32747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:54.155464 32747 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40621
I20260808 03:02:54.155522 393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40621 every 8 connection(s)
I20260808 03:02:54.156968 32747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:54.161085 397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:54.166734 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 32747
I20260808 03:02:54.166970 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:54.167455 822 external_mini_cluster.cc:1428] Setting key 9556e8083734a4817c9068ee2b88600d
I20260808 03:02:54.173015 397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917: Bootstrap starting.
I20260808 03:02:54.177073 397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:54.178469 397 log.cc:824] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:54.182252 397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917: No bootstrap required, opened a new log
Aug 08 03:02:54 dist-test-slave-6ntk krb5kdc[32729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 03:02:54.192720 397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72575dc16841b8a1b5659f85f4d917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40621 } }
I20260808 03:02:54.193266 397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:54.193378 397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a72575dc16841b8a1b5659f85f4d917, State: Initialized, Role: FOLLOWER
I20260808 03:02:54.194304 397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72575dc16841b8a1b5659f85f4d917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40621 } }
I20260808 03:02:54.194607 397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:54.194772 397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:54.194981 397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:54.198508 397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72575dc16841b8a1b5659f85f4d917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40621 } }
I20260808 03:02:54.199210 397 leader_election.cc:302] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a72575dc16841b8a1b5659f85f4d917; no voters:
I20260808 03:02:54.200038 397 leader_election.cc:288] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:54.200160 402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:54.201390 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:54.169531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:55731 (local address 127.0.0.1:40621)
0808 03:02:54.170514 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:54.170530 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:54.170575 (+ 45us) server_negotiation.cc:407] Connection header received
0808 03:02:54.172358 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:54.172389 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:54.172812 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:54.173180 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:54.175104 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:54.176467 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:54.178484 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:54.179505 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:54.179715 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:54.184719 (+ 5004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:54.184727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:54.184745 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:54.184783 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:54.194469 (+ 9686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:54.195818 (+ 1349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:54.195828 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:54.195839 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:54.196125 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:54.196796 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:54.196802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:54.196805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:54.197622 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:54.197986 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:54.199014 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0808 03:02:54.199482 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":453,"threads_started":1}
I20260808 03:02:54.201920 402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [term 1 LEADER]: Becoming Leader. State: Replica: 7a72575dc16841b8a1b5659f85f4d917, State: Running, Role: LEADER
I20260808 03:02:54.202548 402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72575dc16841b8a1b5659f85f4d917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40621 } }
I20260808 03:02:54.206883 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:54.207559 404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a72575dc16841b8a1b5659f85f4d917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72575dc16841b8a1b5659f85f4d917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40621 } } }
I20260808 03:02:54.208019 404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:54.207556 397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:54.207943 403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a72575dc16841b8a1b5659f85f4d917. Latest consensus state: current_term: 1 leader_uuid: "7a72575dc16841b8a1b5659f85f4d917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a72575dc16841b8a1b5659f85f4d917" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40621 } } }
I20260808 03:02:54.209702 403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:54.214924 409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:54.220584 409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:54.242036 409 catalog_manager.cc:1347] Generated new cluster ID: bc6bc0f438a34fac99ff71f471d5fdf5
I20260808 03:02:54.242210 409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:54.254741 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:54.229780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:58891 (local address 127.0.0.1:40621)
0808 03:02:54.230051 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:54.230062 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:54.232792 (+ 2730us) server_negotiation.cc:407] Connection header received
0808 03:02:54.233396 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:54.233406 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:54.233519 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:54.233614 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:54.236686 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:54.238387 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:54.240861 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:54.241637 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:54.241777 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:54.246202 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:54.246211 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:54.246222 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:02:54.246273 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:54.250555 (+ 4282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:54.251763 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:54.251770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:54.251774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:54.251961 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:02:54.252629 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:02:54.252634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:02:54.252638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:02:54.253072 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:54.253333 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:54.254112 (+ 779us) server_negotiation.cc:299] Negotiation successful
0808 03:02:54.254326 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 03:02:54.270608 409 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:54.272068 409 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:54.280589 409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a72575dc16841b8a1b5659f85f4d917: Generated new TSK 0
I20260808 03:02:54.281509 409 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:54.288949 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 32747
I20260808 03:02:54.290230 387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:02:54.384912 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 32747
W20260808 03:02:54.408371 32727 connection.cc:535] client connection to 127.0.0.1:40621 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40621): Connection reset by peer (error 104)
2026-08-08T03:02:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2471 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:54 dist-test-slave-6ntk krb5kdc[439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:54 dist-test-slave-6ntk krb5kdc[439](info): set up 2 sockets
Aug 08 03:02:54 dist-test-slave-6ntk krb5kdc[439](info): commencing operation
krb5kdc: starting...
W20260808 03:02:55.783043 822 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:56 dist-test-slave-6ntk krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:56.137125 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42887
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:56.512099 460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:56.512431 460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:56.512530 460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:56.519939 460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:56.520059 460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:56.520109 460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:56.520177 460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:56.520331 460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:56.531050 460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42887
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:56.532563 460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:56.533989 460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:56.546478 467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:56.546620 472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:56.546658 469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:56.547905 460 server_base.cc:1034] running on GCE node
I20260808 03:02:56.548834 460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:56.550580 460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:56.551841 460 hybrid_clock.cc:648] HybridClock initialized: now 1786158176551762 us; error 81 us; skew 500 ppm
I20260808 03:02:56.552263 460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:56 dist-test-slave-6ntk krb5kdc[439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:56.555655 460 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:56.557257 460 webserver.cc:460] Webserver started at http://127.0.0.1:35423/ using document root <none> and password file <none>
I20260808 03:02:56.558097 460 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:56.558249 460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:56.558561 460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:56.561713 460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "19076ea7ac77456793aeb94346ccfd3c"
format_stamp: "Formatted at 2026-08-08 03:02:56 on dist-test-slave-6ntk"
server_key: "59edb512dc4822c6c0ac9fe8aba12650"
server_key_iv: "7c46f4766028584260631c936f19d44d"
server_key_version: "encryptionkey@0"
I20260808 03:02:56.562600 460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "19076ea7ac77456793aeb94346ccfd3c"
format_stamp: "Formatted at 2026-08-08 03:02:56 on dist-test-slave-6ntk"
server_key: "59edb512dc4822c6c0ac9fe8aba12650"
server_key_iv: "7c46f4766028584260631c936f19d44d"
server_key_version: "encryptionkey@0"
I20260808 03:02:56.567956 460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:02:56.571755 482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:56.573200 460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 03:02:56.573392 460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "19076ea7ac77456793aeb94346ccfd3c"
format_stamp: "Formatted at 2026-08-08 03:02:56 on dist-test-slave-6ntk"
server_key: "59edb512dc4822c6c0ac9fe8aba12650"
server_key_iv: "7c46f4766028584260631c936f19d44d"
server_key_version: "encryptionkey@0"
I20260808 03:02:56.573598 460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:56.597401 460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:56.602133 460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:56.602465 460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:56.623188 460 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42887
I20260808 03:02:56.623226 567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42887 every 8 connection(s)
I20260808 03:02:56.624691 460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:56.628667 568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:56.631479 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 460
I20260808 03:02:56.631668 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:56.632279 822 external_mini_cluster.cc:1428] Setting key 73c79f38f66208ecea86b5c2818b0c7a
I20260808 03:02:56.641425 568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c: Bootstrap starting.
I20260808 03:02:56.644832 568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:56.646278 568 log.cc:824] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:56.646590 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:56.634511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:54933 (local address 127.0.0.1:42887)
0808 03:02:56.635389 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:56.635402 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:56.635450 (+ 48us) server_negotiation.cc:407] Connection header received
0808 03:02:56.637193 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:56.637218 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:56.637674 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:56.638194 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:56.638797 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:56.640193 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:56.641624 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:56.642412 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:56.642620 (+ 208us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:56.644004 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:56.644012 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:56.644027 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:56.644587 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:56.644681 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:56.644708 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:56.644903 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:56.645157 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 03:02:56.645355 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":363,"threads_started":1}
I20260808 03:02:56.649739 568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c: No bootstrap required, opened a new log
I20260808 03:02:56.651356 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:56.658833 568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19076ea7ac77456793aeb94346ccfd3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42887 } }
I20260808 03:02:56.659356 568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:56.659478 568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19076ea7ac77456793aeb94346ccfd3c, State: Initialized, Role: FOLLOWER
I20260808 03:02:56.660223 568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19076ea7ac77456793aeb94346ccfd3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42887 } }
I20260808 03:02:56.660521 568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:56.660665 568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:56.660848 568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:56.663796 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:56.656534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:59367 (local address 127.0.0.1:42887)
0808 03:02:56.656825 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:56.656831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:56.657197 (+ 366us) server_negotiation.cc:407] Connection header received
0808 03:02:56.657381 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:56.657385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:56.657460 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:56.657554 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:56.658300 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:56.659054 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:56.661048 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:02:56.661510 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:02:56.661617 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:02:56.662933 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:56.662937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:56.662940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:56.662963 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:56.663024 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:56.663041 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:56.663131 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:56.663384 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 03:02:56.663431 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 03:02:56.664188 568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19076ea7ac77456793aeb94346ccfd3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42887 } }
I20260808 03:02:56.664776 568 leader_election.cc:302] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19076ea7ac77456793aeb94346ccfd3c; no voters:
I20260808 03:02:56.665496 568 leader_election.cc:288] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:56.665614 579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:56.666435 579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [term 1 LEADER]: Becoming Leader. State: Replica: 19076ea7ac77456793aeb94346ccfd3c, State: Running, Role: LEADER
I20260808 03:02:56.667066 579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19076ea7ac77456793aeb94346ccfd3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42887 } }
I20260808 03:02:56.668113 568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:56.670252 582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19076ea7ac77456793aeb94346ccfd3c. Latest consensus state: current_term: 1 leader_uuid: "19076ea7ac77456793aeb94346ccfd3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19076ea7ac77456793aeb94346ccfd3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42887 } } }
I20260808 03:02:56.670212 580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19076ea7ac77456793aeb94346ccfd3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19076ea7ac77456793aeb94346ccfd3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42887 } } }
I20260808 03:02:56.670764 582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:56.670761 580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:56.672401 587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:56.677918 587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:56.688248 587 catalog_manager.cc:1347] Generated new cluster ID: b7fc58426b68481a8d3dc06907837917
I20260808 03:02:56.688390 587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:56.716084 587 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:56.717971 587 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:56.731640 587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19076ea7ac77456793aeb94346ccfd3c: Generated new TSK 0
I20260808 03:02:56.732801 587 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:56.749025 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 460
W20260808 03:02:56.750494 558 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.193:54933
W20260808 03:02:56.751569 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 460: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:02:56.751748 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 460
W20260808 03:02:56.778851 437 connection.cc:535] client connection to 127.0.0.1:42887 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42887): Connection reset by peer (error 104)
2026-08-08T03:02:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2365 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:56 dist-test-slave-6ntk krb5kdc[611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:56 dist-test-slave-6ntk krb5kdc[611](info): set up 2 sockets
Aug 08 03:02:56 dist-test-slave-6ntk krb5kdc[611](info): commencing operation
krb5kdc: starting...
W20260808 03:02:58.099509 822 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:02:58 dist-test-slave-6ntk krb5kdc[611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:02:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:02:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 03:02:58.445698 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33277
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:02:58.803418 630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:02:58.803723 630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:02:58.803900 630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:02:58.811108 630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:02:58.811220 630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:02:58.811359 630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:02:58.811472 630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:02:58.811563 630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:02:58.821548 630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33277
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:02:58.822993 630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:02:58.824366 630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:02:58.836741 641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:58.836828 639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:02:58.836798 636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:02:58.837703 630 server_base.cc:1034] running on GCE node
I20260808 03:02:58.838634 630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:02:58.840245 630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:02:58.841463 630 hybrid_clock.cc:648] HybridClock initialized: now 1786158178841415 us; error 39 us; skew 500 ppm
I20260808 03:02:58.841822 630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:02:58 dist-test-slave-6ntk krb5kdc[611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158178, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:02:58.849699 630 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 03:02:58.851445 630 webserver.cc:460] Webserver started at http://127.0.0.1:45017/ using document root <none> and password file <none>
I20260808 03:02:58.852205 630 fs_manager.cc:359] Metadata directory not provided
I20260808 03:02:58.852347 630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:02:58.852697 630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:02:58.855978 630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "af865b9d73c04d779d7a3bcce10a6722"
format_stamp: "Formatted at 2026-08-08 03:02:58 on dist-test-slave-6ntk"
server_key: "92171f35ada0923ce57f961139301ef7"
server_key_iv: "d3820318bfebe30463808827df094c4a"
server_key_version: "encryptionkey@0"
I20260808 03:02:58.856787 630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "af865b9d73c04d779d7a3bcce10a6722"
format_stamp: "Formatted at 2026-08-08 03:02:58 on dist-test-slave-6ntk"
server_key: "92171f35ada0923ce57f961139301ef7"
server_key_iv: "d3820318bfebe30463808827df094c4a"
server_key_version: "encryptionkey@0"
I20260808 03:02:58.862265 630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260808 03:02:58.865849 650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:02:58.867292 630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:02:58.867501 630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "af865b9d73c04d779d7a3bcce10a6722"
format_stamp: "Formatted at 2026-08-08 03:02:58 on dist-test-slave-6ntk"
server_key: "92171f35ada0923ce57f961139301ef7"
server_key_iv: "d3820318bfebe30463808827df094c4a"
server_key_version: "encryptionkey@0"
I20260808 03:02:58.867717 630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:02:58.873665 630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:02:58.877655 630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:02:58.877978 630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:02:58.897469 630 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33277
I20260808 03:02:58.897547 724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33277 every 8 connection(s)
I20260808 03:02:58.898998 630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:02:58.900179 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 630
I20260808 03:02:58.900393 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:02:58.900928 822 external_mini_cluster.cc:1428] Setting key b83d351f878ab816cf55bc3b131a34dd
I20260808 03:02:58.903754 727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:02:58.908881 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:58.902825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:35359 (local address 127.0.0.1:33277)
0808 03:02:58.903665 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:58.903677 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:58.903719 (+ 42us) server_negotiation.cc:407] Connection header received
0808 03:02:58.905397 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:58.905421 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:58.905748 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:58.906173 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:58.907285 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:58.907292 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:58.907305 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:58.907318 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:58.907402 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:58.907421 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:58.907600 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:58.907713 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 03:02:58.907918 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":370,"threads_started":1}
I20260808 03:02:58.912773 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:02:58.916873 727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722: Bootstrap starting.
I20260808 03:02:58.920967 727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722: Neither blocks nor log segments found. Creating new log.
I20260808 03:02:58.921197 731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:02:58.918296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:40063 (local address 127.0.0.1:33277)
0808 03:02:58.918502 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 03:02:58.918509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:02:58.919011 (+ 502us) server_negotiation.cc:407] Connection header received
0808 03:02:58.919162 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:02:58.919167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:02:58.919240 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:02:58.919372 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:02:58.920302 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:02:58.920307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:02:58.920312 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:02:58.920332 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:02:58.920404 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:02:58.920426 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:02:58.920549 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 03:02:58.920765 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 03:02:58.920839 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 03:02:58.922413 727 log.cc:824] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722: Log is configured to *not* fsync() on all Append() calls
I20260808 03:02:58.925320 727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722: No bootstrap required, opened a new log
I20260808 03:02:58.932869 727 raft_consensus.cc:348] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af865b9d73c04d779d7a3bcce10a6722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33277 } }
I20260808 03:02:58.933313 727 raft_consensus.cc:374] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:02:58.933434 727 raft_consensus.cc:729] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af865b9d73c04d779d7a3bcce10a6722, State: Initialized, Role: FOLLOWER
I20260808 03:02:58.934105 727 consensus_queue.cc:260] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af865b9d73c04d779d7a3bcce10a6722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33277 } }
I20260808 03:02:58.934358 727 raft_consensus.cc:388] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:02:58.934512 727 raft_consensus.cc:482] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:02:58.934700 727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:02:58.936944 727 raft_consensus.cc:504] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af865b9d73c04d779d7a3bcce10a6722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33277 } }
I20260808 03:02:58.937474 727 leader_election.cc:302] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af865b9d73c04d779d7a3bcce10a6722; no voters:
I20260808 03:02:58.938249 727 leader_election.cc:288] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:02:58.938441 742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:02:58.939199 742 raft_consensus.cc:686] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [term 1 LEADER]: Becoming Leader. State: Replica: af865b9d73c04d779d7a3bcce10a6722, State: Running, Role: LEADER
I20260808 03:02:58.939815 742 consensus_queue.cc:237] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af865b9d73c04d779d7a3bcce10a6722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33277 } }
I20260808 03:02:58.940464 727 sys_catalog.cc:564] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:02:58.943396 745 sys_catalog.cc:455] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af865b9d73c04d779d7a3bcce10a6722" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af865b9d73c04d779d7a3bcce10a6722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33277 } } }
I20260808 03:02:58.943392 746 sys_catalog.cc:455] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af865b9d73c04d779d7a3bcce10a6722. Latest consensus state: current_term: 1 leader_uuid: "af865b9d73c04d779d7a3bcce10a6722" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af865b9d73c04d779d7a3bcce10a6722" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33277 } } }
I20260808 03:02:58.943928 745 sys_catalog.cc:458] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:58.943928 746 sys_catalog.cc:458] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722 [sys.catalog]: This master's current role is: LEADER
I20260808 03:02:58.949651 752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:02:58.955344 752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:02:58.965363 752 catalog_manager.cc:1347] Generated new cluster ID: 5c992b1c0a38471e8052aeadb83e16cd
I20260808 03:02:58.965552 752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:02:58.991071 752 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:02:58.991668 752 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:02:59.004554 752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af865b9d73c04d779d7a3bcce10a6722: Generated new TSK 0
I20260808 03:02:59.005559 752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:02:59.016563 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 630
W20260808 03:02:59.018186 716 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.193:35359
W20260808 03:02:59.019001 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 630: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:02:59.019167 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 630
2026-08-08T03:02:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2268 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:02:59 dist-test-slave-6ntk krb5kdc[774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:02:59 dist-test-slave-6ntk krb5kdc[774](info): set up 2 sockets
Aug 08 03:02:59 dist-test-slave-6ntk krb5kdc[774](info): commencing operation
krb5kdc: starting...
W20260808 03:03:00.402567 822 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:03:00 dist-test-slave-6ntk krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:03:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:03:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:03:00.754806 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:44417
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:44417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:03:01.117952 797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:01.118281 797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:01.118453 797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:01.125982 797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:01.126108 797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:01.126202 797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:01.126323 797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:01.126392 797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:01.136732 797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:44417
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:44417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:01.138202 797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:01.139599 797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:01.152388 803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:01.152452 807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:01.152493 805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:01.152935 797 server_base.cc:1034] running on GCE node
I20260808 03:03:01.154088 797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:01.155673 797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:01.156936 797 hybrid_clock.cc:648] HybridClock initialized: now 1786158181156874 us; error 55 us; skew 500 ppm
I20260808 03:03:01.157331 797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:01 dist-test-slave-6ntk krb5kdc[774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158181, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:01.160905 797 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:03:01.162670 797 webserver.cc:460] Webserver started at http://127.0.205.190:41799/ using document root <none> and password file <none>
I20260808 03:03:01.163425 797 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:01.163589 797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:01.163926 797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:01.167109 797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "972b707ac72443648828313dbfd21ea7"
format_stamp: "Formatted at 2026-08-08 03:03:01 on dist-test-slave-6ntk"
server_key: "ca34f1973528a8bf8aec88b31cc91d81"
server_key_iv: "11e1d1ec8c272ff866b09036ea0bb5c1"
server_key_version: "encryptionkey@0"
I20260808 03:03:01.167935 797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "972b707ac72443648828313dbfd21ea7"
format_stamp: "Formatted at 2026-08-08 03:03:01 on dist-test-slave-6ntk"
server_key: "ca34f1973528a8bf8aec88b31cc91d81"
server_key_iv: "11e1d1ec8c272ff866b09036ea0bb5c1"
server_key_version: "encryptionkey@0"
I20260808 03:03:01.173799 797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 03:03:01.177626 818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:01.179162 797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:03:01.179415 797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "972b707ac72443648828313dbfd21ea7"
format_stamp: "Formatted at 2026-08-08 03:03:01 on dist-test-slave-6ntk"
server_key: "ca34f1973528a8bf8aec88b31cc91d81"
server_key_iv: "11e1d1ec8c272ff866b09036ea0bb5c1"
server_key_version: "encryptionkey@0"
I20260808 03:03:01.179669 797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:01.195715 797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:01.200440 797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:01.200785 797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:01.221141 797 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:44417
I20260808 03:03:01.221148 902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:44417 every 8 connection(s)
I20260808 03:03:01.222718 797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:03:01.226737 903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:01.230654 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 797
I20260808 03:03:01.230854 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:03:01.231333 822 external_mini_cluster.cc:1428] Setting key e01edbbd1f028295a0c6a29936e337ab
I20260808 03:03:01.240574 903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7: Bootstrap starting.
I20260808 03:03:01.244460 903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:01.245530 903 log.cc:824] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7: Log is configured to *not* fsync() on all Append() calls
Aug 08 03:03:01 dist-test-slave-6ntk krb5kdc[774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:03:01.248941 903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7: No bootstrap required, opened a new log
I20260808 03:03:01.256482 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:01.233337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58263 (local address 127.0.205.190:44417)
0808 03:03:01.234180 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:01.234197 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:01.234252 (+ 55us) server_negotiation.cc:407] Connection header received
0808 03:03:01.236172 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:01.236207 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:01.236779 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:01.237253 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:01.238771 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:01.240785 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:01.242197 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:01.243264 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:01.243457 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:01.248277 (+ 4820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:01.248285 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:01.248299 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:01.248344 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:01.251691 (+ 3347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:01.252727 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:01.252735 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:01.252744 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:01.252931 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:01.253403 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:01.253406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:01.253408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:01.254056 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:01.254358 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:01.254895 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 03:03:01.255321 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":368,"threads_started":1}
I20260808 03:03:01.258502 903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b707ac72443648828313dbfd21ea7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44417 } }
I20260808 03:03:01.258924 903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:03:01.259063 903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972b707ac72443648828313dbfd21ea7, State: Initialized, Role: FOLLOWER
I20260808 03:03:01.259805 903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b707ac72443648828313dbfd21ea7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44417 } }
I20260808 03:03:01.260120 903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:03:01.260281 903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:03:01.260496 903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:01.261348 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:03:01.263765 903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b707ac72443648828313dbfd21ea7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44417 } }
I20260808 03:03:01.264464 903 leader_election.cc:302] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 972b707ac72443648828313dbfd21ea7; no voters:
I20260808 03:03:01.265314 903 leader_election.cc:288] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:03:01.265455 911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:03:01.267362 911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [term 1 LEADER]: Becoming Leader. State: Replica: 972b707ac72443648828313dbfd21ea7, State: Running, Role: LEADER
I20260808 03:03:01.268114 911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b707ac72443648828313dbfd21ea7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44417 } }
I20260808 03:03:01.268537 903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:01.271970 922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 972b707ac72443648828313dbfd21ea7. Latest consensus state: current_term: 1 leader_uuid: "972b707ac72443648828313dbfd21ea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b707ac72443648828313dbfd21ea7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44417 } } }
I20260808 03:03:01.275064 922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:01.273108 916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "972b707ac72443648828313dbfd21ea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b707ac72443648828313dbfd21ea7" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 44417 } } }
I20260808 03:03:01.275828 916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:01.277305 933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:03:01.283725 933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:03:01.291319 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:01.269396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55137 (local address 127.0.205.190:44417)
0808 03:03:01.269600 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:01.269610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:01.270133 (+ 523us) server_negotiation.cc:407] Connection header received
0808 03:03:01.270492 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:01.270500 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:01.270609 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:01.270708 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:01.273554 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:01.275264 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:01.277808 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:01.278836 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:01.279010 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:01.282755 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:01.282763 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:01.282769 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:01.282813 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:01.287515 (+ 4702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:01.288641 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:01.288652 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:01.288656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:01.288808 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:01.289376 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:01.289381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:01.289383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:01.289765 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:01.290088 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:01.290567 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 03:03:01.290764 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 03:03:01.294938 933 catalog_manager.cc:1347] Generated new cluster ID: 5d3c3efdce5a4a73b81a39a938ddd57e
I20260808 03:03:01.295068 933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:03:01.337908 933 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:03:01.339270 933 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:03:01.346341 933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 972b707ac72443648828313dbfd21ea7: Generated new TSK 0
I20260808 03:03:01.347265 933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:03:01.358969 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 797
I20260808 03:03:01.360239 898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:03:01.458504 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 797
W20260808 03:03:01.488158 771 connection.cc:535] client connection to 127.0.205.190:44417 recv error: Network error: failed to read from TLS socket (remote: 127.0.205.190:44417): Connection reset by peer (error 104)
2026-08-08T03:03:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:03:01 dist-test-slave-6ntk krb5kdc[956](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:03:01 dist-test-slave-6ntk krb5kdc[956](info): set up 2 sockets
Aug 08 03:03:01 dist-test-slave-6ntk krb5kdc[956](info): commencing operation
krb5kdc: starting...
W20260808 03:03:02.813025 822 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:03:03 dist-test-slave-6ntk krb5kdc[956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:03:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:03:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:03:03.170116 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:40165
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:40165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:03:03.545569 984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:03.546062 984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:03.546237 984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:03.559010 984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:03.559194 984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:03.559280 984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:03.559372 984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:03.559474 984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:03.571959 984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:40165
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:40165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:03.573477 984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:03.574960 984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:03.587837 995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:03.587747 992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:03.587808 991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:03.588691 984 server_base.cc:1034] running on GCE node
I20260808 03:03:03.589488 984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:03.590974 984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:03.592228 984 hybrid_clock.cc:648] HybridClock initialized: now 1786158183592178 us; error 41 us; skew 500 ppm
I20260808 03:03:03.592607 984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:03 dist-test-slave-6ntk krb5kdc[956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158183, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:03.595938 984 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:03:03.597509 984 webserver.cc:460] Webserver started at http://127.0.205.190:33921/ using document root <none> and password file <none>
I20260808 03:03:03.598336 984 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:03.598490 984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:03.598825 984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:03.602078 984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "aba71285059b44a3972565c0fb17869a"
format_stamp: "Formatted at 2026-08-08 03:03:03 on dist-test-slave-6ntk"
server_key: "cd28bcf1cb03990eb344546356b0d58f"
server_key_iv: "3125d93c9ec973d595dfef36bd0b6e3d"
server_key_version: "encryptionkey@0"
I20260808 03:03:03.602967 984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "aba71285059b44a3972565c0fb17869a"
format_stamp: "Formatted at 2026-08-08 03:03:03 on dist-test-slave-6ntk"
server_key: "cd28bcf1cb03990eb344546356b0d58f"
server_key_iv: "3125d93c9ec973d595dfef36bd0b6e3d"
server_key_version: "encryptionkey@0"
I20260808 03:03:03.608471 984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 03:03:03.612077 1008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:03.613394 984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 03:03:03.613592 984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "aba71285059b44a3972565c0fb17869a"
format_stamp: "Formatted at 2026-08-08 03:03:03 on dist-test-slave-6ntk"
server_key: "cd28bcf1cb03990eb344546356b0d58f"
server_key_iv: "3125d93c9ec973d595dfef36bd0b6e3d"
server_key_version: "encryptionkey@0"
I20260808 03:03:03.613837 984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:03.631642 984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:03.636034 984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:03.636354 984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:03.656119 984 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:40165
I20260808 03:03:03.656152 1090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:40165 every 8 connection(s)
I20260808 03:03:03.657538 984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:03:03.661478 1091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:03.665652 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 984
I20260808 03:03:03.665930 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:03:03.666582 822 external_mini_cluster.cc:1428] Setting key e70296dbe129b324996e7e497c9affa5
I20260808 03:03:03.678550 1091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a: Bootstrap starting.
I20260808 03:03:03.682974 1091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:03.684422 1091 log.cc:824] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a: Log is configured to *not* fsync() on all Append() calls
I20260808 03:03:03.685323 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:03.669775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41811 (local address 127.0.205.190:40165)
0808 03:03:03.670790 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:03.670809 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:03.670882 (+ 73us) server_negotiation.cc:407] Connection header received
0808 03:03:03.673129 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:03.673154 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:03.673646 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:03.674172 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:03.675298 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:03.677051 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:03.679259 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:03.680301 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:03.680545 (+ 244us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:03.682110 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:03.682121 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:03.682142 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:03:03.682901 (+ 759us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:03.683016 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:03:03.683053 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:03.683238 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:03.683704 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 03:03:03.683929 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":412,"threads_started":1}
I20260808 03:03:03.688093 1091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a: No bootstrap required, opened a new log
I20260808 03:03:03.690027 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:03:03.699451 1091 raft_consensus.cc:348] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba71285059b44a3972565c0fb17869a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40165 } }
I20260808 03:03:03.699995 1091 raft_consensus.cc:374] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:03:03.700119 1091 raft_consensus.cc:729] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba71285059b44a3972565c0fb17869a, State: Initialized, Role: FOLLOWER
I20260808 03:03:03.700803 1091 consensus_queue.cc:260] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba71285059b44a3972565c0fb17869a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40165 } }
I20260808 03:03:03.701071 1091 raft_consensus.cc:388] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:03:03.701256 1091 raft_consensus.cc:482] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:03:03.701457 1091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:03.703454 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:03.696031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37305 (local address 127.0.205.190:40165)
0808 03:03:03.696275 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:03.696283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:03.696605 (+ 322us) server_negotiation.cc:407] Connection header received
0808 03:03:03.696824 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:03.696828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:03.696909 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:03.696999 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:03.697618 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:03.698581 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:03.700477 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:03.701003 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:03.701140 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:03.702504 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:03.702509 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:03.702512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:03:03.702534 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:03.702574 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:03:03.702593 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:03.702733 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:03.702979 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 03:03:03.703050 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 03:03:03.704644 1091 raft_consensus.cc:504] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba71285059b44a3972565c0fb17869a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40165 } }
I20260808 03:03:03.705293 1091 leader_election.cc:302] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aba71285059b44a3972565c0fb17869a; no voters:
I20260808 03:03:03.706060 1091 leader_election.cc:288] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:03:03.706226 1104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:03:03.706916 1104 raft_consensus.cc:686] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [term 1 LEADER]: Becoming Leader. State: Replica: aba71285059b44a3972565c0fb17869a, State: Running, Role: LEADER
I20260808 03:03:03.707567 1104 consensus_queue.cc:237] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba71285059b44a3972565c0fb17869a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40165 } }
I20260808 03:03:03.708432 1091 sys_catalog.cc:564] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:03.710505 1106 sys_catalog.cc:455] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [sys.catalog]: SysCatalogTable state changed. Reason: New leader aba71285059b44a3972565c0fb17869a. Latest consensus state: current_term: 1 leader_uuid: "aba71285059b44a3972565c0fb17869a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba71285059b44a3972565c0fb17869a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40165 } } }
I20260808 03:03:03.710983 1106 sys_catalog.cc:458] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:03.711836 1105 sys_catalog.cc:455] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aba71285059b44a3972565c0fb17869a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aba71285059b44a3972565c0fb17869a" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 40165 } } }
I20260808 03:03:03.712208 1105 sys_catalog.cc:458] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:03.715157 1112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:03:03.720256 1112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:03:03.729228 1112 catalog_manager.cc:1347] Generated new cluster ID: 79d810011a1a4423a498e59f4ede4535
I20260808 03:03:03.729367 1112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:03:03.738085 1112 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:03:03.739601 1112 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:03:03.746572 1112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aba71285059b44a3972565c0fb17869a: Generated new TSK 0
I20260808 03:03:03.747601 1112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:03:03.763916 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 984
W20260808 03:03:03.765262 1086 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41811
W20260808 03:03:03.766146 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 984: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:03:03.766305 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 984
W20260808 03:03:03.790305 951 connection.cc:535] client connection to 127.0.205.190:40165 recv error: Network error: failed to read from TLS socket (remote: 127.0.205.190:40165): Connection reset by peer (error 104)
2026-08-08T03:03:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:03:03 dist-test-slave-6ntk krb5kdc[1142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:03:03 dist-test-slave-6ntk krb5kdc[1142](info): set up 2 sockets
Aug 08 03:03:03 dist-test-slave-6ntk krb5kdc[1142](info): commencing operation
krb5kdc: starting...
W20260808 03:03:05.127166 822 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.002s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:03:05 dist-test-slave-6ntk krb5kdc[1142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:03:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:03:05.489485 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:42367
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:36091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:42367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:03:05.857101 1164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:05.857422 1164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:05.857606 1164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:05.865069 1164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:05.865211 1164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:05.865307 1164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:05.865427 1164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:05.865499 1164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:05.876036 1164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:36091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:42367
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:42367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:05.877565 1164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:05.878937 1164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:05.891767 1171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:05.891723 1172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:05.891628 1177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:05.892764 1164 server_base.cc:1034] running on GCE node
I20260808 03:03:05.893704 1164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:05.895264 1164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:05.896528 1164 hybrid_clock.cc:648] HybridClock initialized: now 1786158185896479 us; error 43 us; skew 500 ppm
I20260808 03:03:05.896900 1164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:05 dist-test-slave-6ntk krb5kdc[1142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158185, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:05.900332 1164 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:03:05.902045 1164 webserver.cc:460] Webserver started at http://127.0.205.190:44243/ using document root <none> and password file <none>
I20260808 03:03:05.902835 1164 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:05.902987 1164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:05.903313 1164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:05.906540 1164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "7c839646387c41a88641d2f9e02bb179"
format_stamp: "Formatted at 2026-08-08 03:03:05 on dist-test-slave-6ntk"
server_key: "ecbdf51e97067322b680c953b87f17ed"
server_key_iv: "fe278940af8de2149d879fbf1383abdc"
server_key_version: "encryptionkey@0"
I20260808 03:03:05.907366 1164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "7c839646387c41a88641d2f9e02bb179"
format_stamp: "Formatted at 2026-08-08 03:03:05 on dist-test-slave-6ntk"
server_key: "ecbdf51e97067322b680c953b87f17ed"
server_key_iv: "fe278940af8de2149d879fbf1383abdc"
server_key_version: "encryptionkey@0"
I20260808 03:03:05.913403 1164 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260808 03:03:05.917167 1185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:05.918598 1164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:03:05.918805 1164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "7c839646387c41a88641d2f9e02bb179"
format_stamp: "Formatted at 2026-08-08 03:03:05 on dist-test-slave-6ntk"
server_key: "ecbdf51e97067322b680c953b87f17ed"
server_key_iv: "fe278940af8de2149d879fbf1383abdc"
server_key_version: "encryptionkey@0"
I20260808 03:03:05.919041 1164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:05.925092 1164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:05.929773 1164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:05.930126 1164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:05.949921 1164 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:42367
I20260808 03:03:05.949970 1266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:42367 every 8 connection(s)
I20260808 03:03:05.951429 1164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:03:05.954012 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1164
I20260808 03:03:05.954190 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:03:05.954653 822 external_mini_cluster.cc:1428] Setting key c697df34bd2c59089caae37992553dc7
I20260808 03:03:05.955612 1267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:05.963299 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:05.956585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34463 (local address 127.0.205.190:42367)
0808 03:03:05.957466 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:05.957482 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:05.957532 (+ 50us) server_negotiation.cc:407] Connection header received
0808 03:03:05.959343 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:05.959380 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:05.959829 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:05.960272 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:05.961271 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:05.961279 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:05.961296 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:03:05.961313 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:05.961428 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:03:05.961456 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:05.961669 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:05.961806 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 03:03:05.962029 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":336,"threads_started":1}
I20260808 03:03:05.967424 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:03:05.970134 1267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179: Bootstrap starting.
I20260808 03:03:05.974009 1267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:05.974772 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:05.972056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39723 (local address 127.0.205.190:42367)
0808 03:03:05.972258 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:05.972264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:05.972672 (+ 408us) server_negotiation.cc:407] Connection header received
0808 03:03:05.972823 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:05.972829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:05.972892 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:05.973008 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:05.974072 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:05.974077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:05.974084 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:03:05.974106 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:05.974162 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:03:05.974185 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:05.974311 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:05.974405 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 03:03:05.974481 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 03:03:05.975412 1267 log.cc:824] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179: Log is configured to *not* fsync() on all Append() calls
I20260808 03:03:05.978297 1267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179: No bootstrap required, opened a new log
I20260808 03:03:05.985564 1267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c839646387c41a88641d2f9e02bb179" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42367 } }
I20260808 03:03:05.986068 1267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:03:05.986186 1267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c839646387c41a88641d2f9e02bb179, State: Initialized, Role: FOLLOWER
I20260808 03:03:05.986773 1267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c839646387c41a88641d2f9e02bb179" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42367 } }
I20260808 03:03:05.987030 1267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:03:05.987186 1267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:03:05.987370 1267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:05.989611 1267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c839646387c41a88641d2f9e02bb179" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42367 } }
I20260808 03:03:05.990219 1267 leader_election.cc:302] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c839646387c41a88641d2f9e02bb179; no voters:
I20260808 03:03:05.990995 1267 leader_election.cc:288] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:03:05.991112 1287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:03:05.991839 1287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [term 1 LEADER]: Becoming Leader. State: Replica: 7c839646387c41a88641d2f9e02bb179, State: Running, Role: LEADER
I20260808 03:03:05.992489 1287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c839646387c41a88641d2f9e02bb179" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42367 } }
I20260808 03:03:05.993417 1267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:05.995522 1288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c839646387c41a88641d2f9e02bb179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c839646387c41a88641d2f9e02bb179" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42367 } } }
I20260808 03:03:05.995983 1288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:05.996378 1289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c839646387c41a88641d2f9e02bb179. Latest consensus state: current_term: 1 leader_uuid: "7c839646387c41a88641d2f9e02bb179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c839646387c41a88641d2f9e02bb179" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42367 } } }
I20260808 03:03:05.996810 1289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:05.998448 1299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:03:06.003883 1299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:03:06.012847 1299 catalog_manager.cc:1347] Generated new cluster ID: 120c10aca3b249c3964a77afdf74dda8
I20260808 03:03:06.012990 1299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:03:06.033427 1299 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:03:06.033916 1299 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:03:06.047163 1299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c839646387c41a88641d2f9e02bb179: Generated new TSK 0
I20260808 03:03:06.048108 1299 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:03:06.057402 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1164
W20260808 03:03:06.058808 1258 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34463
W20260808 03:03:06.059572 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1164: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:03:06.059736 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1164
2026-08-08T03:03:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:03:06 dist-test-slave-6ntk krb5kdc[1316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:03:06 dist-test-slave-6ntk krb5kdc[1316](info): set up 2 sockets
Aug 08 03:03:06 dist-test-slave-6ntk krb5kdc[1316](info): commencing operation
krb5kdc: starting...
W20260808 03:03:07.415230 822 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:03:07 dist-test-slave-6ntk krb5kdc[1316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:03:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:03:07.763417 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:42845
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:45251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:42845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:03:08.119012 1334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:08.119501 1334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:08.119679 1334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:08.127115 1334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:08.127238 1334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:08.127337 1334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:08.127442 1334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:08.127509 1334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:08.137809 1334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:45251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:42845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:42845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:08.139268 1334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:08.140652 1334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:08.153154 1340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:08.153219 1343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:08.153240 1347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:08.154152 1334 server_base.cc:1034] running on GCE node
I20260808 03:03:08.155064 1334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:08.156539 1334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:08.157775 1334 hybrid_clock.cc:648] HybridClock initialized: now 1786158188157720 us; error 52 us; skew 500 ppm
I20260808 03:03:08.158176 1334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:08 dist-test-slave-6ntk krb5kdc[1316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158188, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:08.161409 1334 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:03:08.163106 1334 webserver.cc:460] Webserver started at http://127.0.205.190:34307/ using document root <none> and password file <none>
I20260808 03:03:08.163848 1334 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:08.164011 1334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:08.164331 1334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:08.167474 1334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "086ccad64e164ddf903d1b9e3b72c272"
format_stamp: "Formatted at 2026-08-08 03:03:08 on dist-test-slave-6ntk"
server_key: "b5fe26e9cdc7c92366b585350e309dfb"
server_key_iv: "ded0bcb8b91f9be937f322d37485792b"
server_key_version: "encryptionkey@0"
I20260808 03:03:08.168296 1334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "086ccad64e164ddf903d1b9e3b72c272"
format_stamp: "Formatted at 2026-08-08 03:03:08 on dist-test-slave-6ntk"
server_key: "b5fe26e9cdc7c92366b585350e309dfb"
server_key_iv: "ded0bcb8b91f9be937f322d37485792b"
server_key_version: "encryptionkey@0"
I20260808 03:03:08.173998 1334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 03:03:08.177693 1355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:08.179112 1334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:03:08.179411 1334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "086ccad64e164ddf903d1b9e3b72c272"
format_stamp: "Formatted at 2026-08-08 03:03:08 on dist-test-slave-6ntk"
server_key: "b5fe26e9cdc7c92366b585350e309dfb"
server_key_iv: "ded0bcb8b91f9be937f322d37485792b"
server_key_version: "encryptionkey@0"
I20260808 03:03:08.179625 1334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:08.198171 1334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:08.202273 1334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:08.202584 1334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:08.222215 1334 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:42845
I20260808 03:03:08.222236 1431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:42845 every 8 connection(s)
I20260808 03:03:08.223721 1334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:03:08.227805 1433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:08.228524 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1334
I20260808 03:03:08.228715 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:03:08.229322 822 external_mini_cluster.cc:1428] Setting key 9fd40cc3e7ede3094c9faf1f241ab7d1
I20260808 03:03:08.242640 1433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272: Bootstrap starting.
Aug 08 03:03:08 dist-test-slave-6ntk krb5kdc[1316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:03:08.246788 1433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:08.248299 1433 log.cc:824] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272: Log is configured to *not* fsync() on all Append() calls
I20260808 03:03:08.251820 1433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272: No bootstrap required, opened a new log
I20260808 03:03:08.255702 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:08.231232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58719 (local address 127.0.205.190:42845)
0808 03:03:08.232063 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:08.232075 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:08.232118 (+ 43us) server_negotiation.cc:407] Connection header received
0808 03:03:08.234450 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:08.234476 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:08.234949 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:08.235298 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:08.237196 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:08.238587 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:08.240541 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:08.241231 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:08.241372 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:08.246475 (+ 5103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:08.246489 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:08.246511 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:08.246564 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:08.250583 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:08.251449 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:08.251457 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:08.251465 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:08.251681 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:08.252262 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:08.252268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:08.252271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:08.253043 (+ 772us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:08.253374 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:08.253696 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 03:03:08.254250 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":395,"threads_started":1}
I20260808 03:03:08.260227 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:03:08.262979 1433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ccad64e164ddf903d1b9e3b72c272" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42845 } }
I20260808 03:03:08.263530 1433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:03:08.263672 1433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086ccad64e164ddf903d1b9e3b72c272, State: Initialized, Role: FOLLOWER
I20260808 03:03:08.264485 1433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ccad64e164ddf903d1b9e3b72c272" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42845 } }
I20260808 03:03:08.264806 1433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:03:08.264961 1433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:03:08.265141 1433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:08.268929 1433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ccad64e164ddf903d1b9e3b72c272" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42845 } }
I20260808 03:03:08.269613 1433 leader_election.cc:302] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 086ccad64e164ddf903d1b9e3b72c272; no voters:
I20260808 03:03:08.270437 1433 leader_election.cc:288] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:03:08.270560 1452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:03:08.272301 1452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [term 1 LEADER]: Becoming Leader. State: Replica: 086ccad64e164ddf903d1b9e3b72c272, State: Running, Role: LEADER
I20260808 03:03:08.273094 1452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ccad64e164ddf903d1b9e3b72c272" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42845 } }
I20260808 03:03:08.273342 1433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:08.283911 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:08.266009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36629 (local address 127.0.205.190:42845)
0808 03:03:08.266201 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:08.266212 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:08.266770 (+ 558us) server_negotiation.cc:407] Connection header received
0808 03:03:08.266967 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:08.266971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:08.267041 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:08.267124 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:08.269264 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:08.270234 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:08.272037 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:08.272771 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:08.272929 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:08.276070 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:08.276076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:08.276081 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:08.276126 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:08.279440 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:08.280500 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:08.280504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:08.280507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:08.280662 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:08.281162 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:08.281166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:08.281169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:08.281578 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:08.281822 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:08.283139 (+ 1317us) server_negotiation.cc:299] Negotiation successful
0808 03:03:08.283421 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 03:03:08.284312 1453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "086ccad64e164ddf903d1b9e3b72c272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ccad64e164ddf903d1b9e3b72c272" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42845 } } }
I20260808 03:03:08.284758 1453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:08.285241 1455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 086ccad64e164ddf903d1b9e3b72c272. Latest consensus state: current_term: 1 leader_uuid: "086ccad64e164ddf903d1b9e3b72c272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ccad64e164ddf903d1b9e3b72c272" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42845 } } }
I20260808 03:03:08.285643 1455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:08.286126 1472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:03:08.290894 1472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:03:08.300266 1472 catalog_manager.cc:1347] Generated new cluster ID: 7b854ec99b0546399f11e75b12edb979
I20260808 03:03:08.300386 1472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:03:08.320043 1472 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:03:08.321393 1472 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:03:08.328780 1472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 086ccad64e164ddf903d1b9e3b72c272: Generated new TSK 0
I20260808 03:03:08.329661 1472 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:03:08.337745 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1334
I20260808 03:03:08.339111 1424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:03:08.430974 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1334
2026-08-08T03:03:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2367 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:03:08 dist-test-slave-6ntk krb5kdc[1487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:03:08 dist-test-slave-6ntk krb5kdc[1487](info): set up 2 sockets
Aug 08 03:03:08 dist-test-slave-6ntk krb5kdc[1487](info): commencing operation
krb5kdc: starting...
W20260808 03:03:09.863979 822 mini_kdc.cc:121] Time spent starting KDC: real 1.389s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:03:10 dist-test-slave-6ntk krb5kdc[1487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:03:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:03:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:03:10.222462 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43613
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:39259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:43613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:03:10.610357 1506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:10.610685 1506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:10.610790 1506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:10.618119 1506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:10.618247 1506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:10.618301 1506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:10.618364 1506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:10.618408 1506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:10.628947 1506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:39259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:43613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:43613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:10.630438 1506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:10.631783 1506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:10.644791 1515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:10.644711 1517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:10.644788 1520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:10.645603 1506 server_base.cc:1034] running on GCE node
I20260808 03:03:10.646544 1506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:10.648093 1506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:10.649336 1506 hybrid_clock.cc:648] HybridClock initialized: now 1786158190649289 us; error 38 us; skew 500 ppm
I20260808 03:03:10.649746 1506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:10 dist-test-slave-6ntk krb5kdc[1487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158190, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:10.657487 1506 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:03:10.659193 1506 webserver.cc:460] Webserver started at http://127.0.205.190:32979/ using document root <none> and password file <none>
I20260808 03:03:10.659914 1506 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:10.660023 1506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:10.660398 1506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:10.663534 1506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b"
format_stamp: "Formatted at 2026-08-08 03:03:10 on dist-test-slave-6ntk"
server_key: "50c16c850ee78d653f9604f37420ebdd"
server_key_iv: "7768afcf9c064a8e9944ef9d8718b2be"
server_key_version: "encryptionkey@0"
I20260808 03:03:10.664253 1506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b"
format_stamp: "Formatted at 2026-08-08 03:03:10 on dist-test-slave-6ntk"
server_key: "50c16c850ee78d653f9604f37420ebdd"
server_key_iv: "7768afcf9c064a8e9944ef9d8718b2be"
server_key_version: "encryptionkey@0"
I20260808 03:03:10.669704 1506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 03:03:10.673594 1531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:10.675513 1506 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 03:03:10.675760 1506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b"
format_stamp: "Formatted at 2026-08-08 03:03:10 on dist-test-slave-6ntk"
server_key: "50c16c850ee78d653f9604f37420ebdd"
server_key_iv: "7768afcf9c064a8e9944ef9d8718b2be"
server_key_version: "encryptionkey@0"
I20260808 03:03:10.675982 1506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:10.702992 1506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:10.708086 1506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:10.708443 1506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:10.729348 1506 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:43613
I20260808 03:03:10.729344 1605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:43613 every 8 connection(s)
I20260808 03:03:10.730997 1506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:03:10.731968 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1506
I20260808 03:03:10.732177 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:03:10.732765 822 external_mini_cluster.cc:1428] Setting key 7aeb46af24cda74f15bc2ed95e0ac1f7
I20260808 03:03:10.736685 1606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:10.749416 1610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:10.734842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59691 (local address 127.0.205.190:43613)
0808 03:03:10.738427 (+ 3585us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:10.738440 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:10.738485 (+ 45us) server_negotiation.cc:407] Connection header received
0808 03:03:10.740219 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:10.740243 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:10.740643 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:10.741166 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:10.741753 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:10.743189 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:10.744567 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:10.745273 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:10.745489 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:10.746942 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:10.746949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:10.746964 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:03:10.747606 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:10.747696 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:03:10.747726 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:10.747863 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:10.748131 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 03:03:10.748423 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3364,"thread_start_us":2947,"threads_started":1}
I20260808 03:03:10.749715 1606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b: Bootstrap starting.
I20260808 03:03:10.753469 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:03:10.753500 1606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:10.754624 1606 log.cc:824] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b: Log is configured to *not* fsync() on all Append() calls
I20260808 03:03:10.757977 1606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b: No bootstrap required, opened a new log
I20260808 03:03:10.765512 1610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:10.758376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52219 (local address 127.0.205.190:43613)
0808 03:03:10.758611 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:10.758620 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:10.759103 (+ 483us) server_negotiation.cc:407] Connection header received
0808 03:03:10.759297 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:10.759301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:10.759364 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:10.759473 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:10.760140 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:10.760925 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:10.762903 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:10.763375 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:10.763498 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:10.764743 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:10.764748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:10.764751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:03:10.764772 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:10.764812 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:03:10.764830 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:10.764902 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:10.765070 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 03:03:10.765152 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:03:10.768232 1606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43613 } }
I20260808 03:03:10.768662 1606 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:03:10.768776 1606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c3bce6ad6f64a8f81309ab0f4cf267b, State: Initialized, Role: FOLLOWER
I20260808 03:03:10.769387 1606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43613 } }
I20260808 03:03:10.769629 1606 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:03:10.769785 1606 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:03:10.770011 1606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:10.772389 1606 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43613 } }
I20260808 03:03:10.772935 1606 leader_election.cc:302] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c3bce6ad6f64a8f81309ab0f4cf267b; no voters:
I20260808 03:03:10.773612 1606 leader_election.cc:288] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:03:10.773757 1619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:03:10.774559 1619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [term 1 LEADER]: Becoming Leader. State: Replica: 6c3bce6ad6f64a8f81309ab0f4cf267b, State: Running, Role: LEADER
I20260808 03:03:10.775197 1619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43613 } }
I20260808 03:03:10.776007 1606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:10.779248 1620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43613 } } }
I20260808 03:03:10.779808 1620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:10.781121 1621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c3bce6ad6f64a8f81309ab0f4cf267b. Latest consensus state: current_term: 1 leader_uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3bce6ad6f64a8f81309ab0f4cf267b" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 43613 } } }
I20260808 03:03:10.781731 1621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:10.782272 1632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:03:10.787159 1632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:03:10.795492 1632 catalog_manager.cc:1347] Generated new cluster ID: 75a67830b8f246768f3f049c4335efb5
I20260808 03:03:10.795619 1632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:03:10.809077 1632 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:03:10.811041 1632 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:03:10.823249 1632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c3bce6ad6f64a8f81309ab0f4cf267b: Generated new TSK 0
I20260808 03:03:10.824183 1632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:03:10.831813 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1506
W20260808 03:03:10.833144 1594 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59691
W20260808 03:03:10.833943 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1506: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:03:10.834105 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1506
2026-08-08T03:03:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:03:10 dist-test-slave-6ntk krb5kdc[1648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:03:10 dist-test-slave-6ntk krb5kdc[1648](info): set up 2 sockets
Aug 08 03:03:10 dist-test-slave-6ntk krb5kdc[1648](info): commencing operation
krb5kdc: starting...
W20260808 03:03:12.175591 822 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:03:12 dist-test-slave-6ntk krb5kdc[1648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:03:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:03:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:03:12.541721 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:38255
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:43627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:38255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:03:12.906972 1668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:12.907368 1668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:12.907548 1668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:12.915175 1668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:12.915371 1668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:12.915494 1668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:12.915606 1668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:12.915685 1668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:12.926227 1668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:43627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:38255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:38255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:12.927625 1668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:12.929008 1668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:12.941900 1675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:12.941943 1679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:12.941964 1674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:12.942615 1668 server_base.cc:1034] running on GCE node
I20260808 03:03:12.943544 1668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:12.945097 1668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:12.946398 1668 hybrid_clock.cc:648] HybridClock initialized: now 1786158192946352 us; error 58 us; skew 500 ppm
I20260808 03:03:12.946799 1668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:12 dist-test-slave-6ntk krb5kdc[1648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158192, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:12.950197 1668 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:03:12.951666 1668 webserver.cc:460] Webserver started at http://127.0.205.190:34825/ using document root <none> and password file <none>
I20260808 03:03:12.952343 1668 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:12.952468 1668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:12.952729 1668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:12.955881 1668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "c610aa7bec7e476ea477ee09045b7655"
format_stamp: "Formatted at 2026-08-08 03:03:12 on dist-test-slave-6ntk"
server_key: "08467c3541df2dcef4bf58857d1524c9"
server_key_iv: "29ba3202f75189c2b2a10bea99be686e"
server_key_version: "encryptionkey@0"
I20260808 03:03:12.956676 1668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "c610aa7bec7e476ea477ee09045b7655"
format_stamp: "Formatted at 2026-08-08 03:03:12 on dist-test-slave-6ntk"
server_key: "08467c3541df2dcef4bf58857d1524c9"
server_key_iv: "29ba3202f75189c2b2a10bea99be686e"
server_key_version: "encryptionkey@0"
I20260808 03:03:12.962425 1668 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 03:03:12.966264 1691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:12.967648 1668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:03:12.967839 1668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "c610aa7bec7e476ea477ee09045b7655"
format_stamp: "Formatted at 2026-08-08 03:03:12 on dist-test-slave-6ntk"
server_key: "08467c3541df2dcef4bf58857d1524c9"
server_key_iv: "29ba3202f75189c2b2a10bea99be686e"
server_key_version: "encryptionkey@0"
I20260808 03:03:12.968065 1668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:12.974313 1668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:12.979053 1668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:12.979390 1668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:12.999333 1668 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:38255
I20260808 03:03:12.999357 1771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:38255 every 8 connection(s)
I20260808 03:03:13.000890 1668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:03:13.004902 1775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:13.010690 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1668
I20260808 03:03:13.010908 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:03:13.011384 822 external_mini_cluster.cc:1428] Setting key 226c561f6bf507e4de9572af573f0ee3
I20260808 03:03:13.016570 1775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655: Bootstrap starting.
I20260808 03:03:13.019935 1782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:13.013466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39971 (local address 127.0.205.190:38255)
0808 03:03:13.014484 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:13.014497 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:13.014549 (+ 52us) server_negotiation.cc:407] Connection header received
0808 03:03:13.016261 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:13.016289 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:13.016642 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:13.017055 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:13.018222 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:13.018228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:13.018241 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:03:13.018332 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:13.018398 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:03:13.018423 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:13.018605 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:13.018754 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 03:03:13.018978 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":521,"threads_started":1}
I20260808 03:03:13.020855 1775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:13.022142 1775 log.cc:824] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655: Log is configured to *not* fsync() on all Append() calls
I20260808 03:03:13.024161 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:03:13.025434 1775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655: No bootstrap required, opened a new log
I20260808 03:03:13.032486 1782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:13.028620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42071 (local address 127.0.205.190:38255)
0808 03:03:13.028783 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:13.028787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:13.030410 (+ 1623us) server_negotiation.cc:407] Connection header received
0808 03:03:13.030626 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:13.030631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:13.030680 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:13.030797 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:13.031775 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:13.031780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:13.031785 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 03:03:13.031804 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:13.031856 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 03:03:13.031872 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:13.032028 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:13.032121 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 03:03:13.032195 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:03:13.035171 1775 raft_consensus.cc:348] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c610aa7bec7e476ea477ee09045b7655" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38255 } }
I20260808 03:03:13.035677 1775 raft_consensus.cc:374] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 03:03:13.035843 1775 raft_consensus.cc:729] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c610aa7bec7e476ea477ee09045b7655, State: Initialized, Role: FOLLOWER
I20260808 03:03:13.036548 1775 consensus_queue.cc:260] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c610aa7bec7e476ea477ee09045b7655" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38255 } }
I20260808 03:03:13.036815 1775 raft_consensus.cc:388] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 03:03:13.036978 1775 raft_consensus.cc:482] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 03:03:13.037159 1775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:13.039516 1775 raft_consensus.cc:504] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c610aa7bec7e476ea477ee09045b7655" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38255 } }
I20260808 03:03:13.040097 1775 leader_election.cc:302] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c610aa7bec7e476ea477ee09045b7655; no voters:
I20260808 03:03:13.040899 1775 leader_election.cc:288] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 03:03:13.041093 1794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:03:13.041853 1794 raft_consensus.cc:686] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [term 1 LEADER]: Becoming Leader. State: Replica: c610aa7bec7e476ea477ee09045b7655, State: Running, Role: LEADER
I20260808 03:03:13.042521 1794 consensus_queue.cc:237] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c610aa7bec7e476ea477ee09045b7655" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38255 } }
I20260808 03:03:13.043145 1775 sys_catalog.cc:564] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:13.046007 1796 sys_catalog.cc:455] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c610aa7bec7e476ea477ee09045b7655. Latest consensus state: current_term: 1 leader_uuid: "c610aa7bec7e476ea477ee09045b7655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c610aa7bec7e476ea477ee09045b7655" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38255 } } }
I20260808 03:03:13.046751 1796 sys_catalog.cc:458] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:13.046025 1795 sys_catalog.cc:455] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c610aa7bec7e476ea477ee09045b7655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c610aa7bec7e476ea477ee09045b7655" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 38255 } } }
I20260808 03:03:13.047461 1795 sys_catalog.cc:458] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:13.048933 1806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:03:13.054574 1806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:03:13.055338 1817 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 03:03:13.064088 1806 catalog_manager.cc:1347] Generated new cluster ID: f029aafd1c4e41328341a793e65badf0
I20260808 03:03:13.064244 1806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:03:13.087180 1806 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:03:13.087749 1806 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:03:13.098811 1806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c610aa7bec7e476ea477ee09045b7655: Generated new TSK 0
I20260808 03:03:13.099776 1806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:03:13.112771 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1668
W20260808 03:03:13.114423 1765 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39971
W20260808 03:03:13.115269 822 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1668: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 03:03:13.115430 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1668
2026-08-08T03:03:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2283 ms)
[----------] 18 tests from AuthTokenIssuingTest (42527 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:03:13 dist-test-slave-6ntk krb5kdc[1822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:03:13 dist-test-slave-6ntk krb5kdc[1822](info): set up 2 sockets
Aug 08 03:03:13 dist-test-slave-6ntk krb5kdc[1822](info): commencing operation
krb5kdc: starting...
W20260808 03:03:14.477399 822 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.003s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:03:14 dist-test-slave-6ntk krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:03:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:03:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:03:14.842423 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:34439
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:36371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:34439,127.0.205.189:38183,127.0.205.188:44785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:03:15.212026 1843 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:03:15.212394 1843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:15.212472 1843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:15.212555 1843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:15.220173 1843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:15.220276 1843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:15.220342 1843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:15.220395 1843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:15.220459 1843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:15.230830 1843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:36371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:34439,127.0.205.189:38183,127.0.205.188:44785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:34439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:15.232177 1843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:15.233536 1843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:15.245630 1850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:15.245663 1851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:15.246018 1854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:15.247031 1843 server_base.cc:1034] running on GCE node
I20260808 03:03:15.247852 1843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:15.249298 1843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:15.250556 1843 hybrid_clock.cc:648] HybridClock initialized: now 1786158195250493 us; error 37 us; skew 500 ppm
I20260808 03:03:15.250900 1843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:15 dist-test-slave-6ntk krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:15.254309 1843 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:03:15.255798 1843 webserver.cc:460] Webserver started at http://127.0.205.190:41679/ using document root <none> and password file <none>
I20260808 03:03:15.256475 1843 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:15.256572 1843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:15.256842 1843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:15.259887 1843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "aeaa82f8aa2a4949a7a88c06c9210c27"
format_stamp: "Formatted at 2026-08-08 03:03:15 on dist-test-slave-6ntk"
server_key: "a3955be3e59fdae515bd825376fe5187"
server_key_iv: "972bdd86f7e01622379741592d706daf"
server_key_version: "encryptionkey@0"
I20260808 03:03:15.260659 1843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "aeaa82f8aa2a4949a7a88c06c9210c27"
format_stamp: "Formatted at 2026-08-08 03:03:15 on dist-test-slave-6ntk"
server_key: "a3955be3e59fdae515bd825376fe5187"
server_key_iv: "972bdd86f7e01622379741592d706daf"
server_key_version: "encryptionkey@0"
I20260808 03:03:15.266064 1843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 03:03:15.269801 1863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:15.271373 1843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:03:15.271528 1843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "aeaa82f8aa2a4949a7a88c06c9210c27"
format_stamp: "Formatted at 2026-08-08 03:03:15 on dist-test-slave-6ntk"
server_key: "a3955be3e59fdae515bd825376fe5187"
server_key_iv: "972bdd86f7e01622379741592d706daf"
server_key_version: "encryptionkey@0"
I20260808 03:03:15.271716 1843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:15.290374 1843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:15.295219 1843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:15.295475 1843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:15.315464 1843 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:34439
I20260808 03:03:15.315485 1938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:34439 every 8 connection(s)
I20260808 03:03:15.316921 1843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:03:15.317509 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1843
I20260808 03:03:15.317751 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:03:15.318619 822 external_mini_cluster.cc:1428] Setting key 89bf71c9cfb5f0cf3f97a8795cd47bad
I20260808 03:03:15.321094 1939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:15.328008 1939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:03:15 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.205.189@KRBTEST.COM; defaulting to no policy
I20260808 03:03:15.356123 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.331515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34439 (local address 127.0.0.1:52538)
0808 03:03:15.332566 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.332624 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:15.335692 (+ 3068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:15.336628 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:15.336637 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:15.337497 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:15.338118 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:15.338147 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.339823 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.339832 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:15.341999 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:15.342019 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.343160 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.343173 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:15.343416 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:15.344201 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:15.344237 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:15.347619 (+ 3382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:15.351347 (+ 3728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:15.351365 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:15.351383 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:15.352097 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:15.352562 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:15.352568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:15.352571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:15.352807 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:15.353662 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:15.353673 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:15.354013 (+ 340us) client_negotiation.cc:769] Sending connection context
0808 03:03:15.354335 (+ 322us) client_negotiation.cc:240] Negotiation successful
0808 03:03:15.355116 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":149,"thread_start_us":434,"threads_started":1}
I20260808 03:03:15.356173 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.331515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52538 (local address 127.0.205.190:34439)
0808 03:03:15.333564 (+ 2049us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:15.333571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:15.333613 (+ 42us) server_negotiation.cc:407] Connection header received
0808 03:03:15.336101 (+ 2488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:15.336107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:15.336349 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:15.336534 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:15.338359 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.339619 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.342160 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.342857 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.343005 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:15.347933 (+ 4928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:15.347943 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:15.347957 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:15.347997 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:15.351153 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:15.352249 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:15.352256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:15.352258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:15.352436 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:15.352938 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:15.352941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:15.352943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:15.353498 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:15.353661 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:15.354476 (+ 815us) server_negotiation.cc:299] Negotiation successful
0808 03:03:15.354936 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":1840,"spinlock_wait_cycles":42752,"thread_start_us":1595,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.189@KRBTEST.COM" created.
I20260808 03:03:15.359304 1939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:03:15.360633 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.360193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37388)
0808 03:03:15.360364 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.360479 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 03:03:15.360905 1867 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
I20260808 03:03:15.362743 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.362259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37398)
0808 03:03:15.362453 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.362552 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 03:03:15.363008 1939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:38183: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab.
Entry for principal kudu/127.0.205.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab.
I20260808 03:03:15.398102 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.189:38183
--webserver_interface=127.0.205.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab
--principal=kudu/127.0.205.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:36371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:34439,127.0.205.189:38183,127.0.205.188:44785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:03:15.412420 1939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } attempt: 1
I20260808 03:03:15.413807 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.413280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37410)
0808 03:03:15.413512 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.413663 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 03:03:15.415024 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.414629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37416)
0808 03:03:15.414764 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.414836 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 03:03:15.415228 1939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:38183: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
I20260808 03:03:15.483515 1939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } attempt: 2
I20260808 03:03:15.484769 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.484297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37422)
0808 03:03:15.484516 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.484622 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 03:03:15.485842 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.485463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37428)
0808 03:03:15.485615 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.485703 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 03:03:15.486150 1939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:38183: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
I20260808 03:03:15.577438 1939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } attempt: 3
I20260808 03:03:15.578959 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.578269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37440)
0808 03:03:15.578544 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.578721 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 03:03:15.580283 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.579725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37444)
0808 03:03:15.579943 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.580058 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 03:03:15.580561 1939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:38183: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
I20260808 03:03:15.723929 1939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } attempt: 4
I20260808 03:03:15.725363 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.724691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37452)
0808 03:03:15.725057 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.725178 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260808 03:03:15.726519 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.726082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37462)
0808 03:03:15.726247 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.726351 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 03:03:15.726773 1939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:38183: Network error: Client connection negotiation failed: client connection to 127.0.205.189:38183: connect: Connection refused (error 111)
W20260808 03:03:15.761652 1951 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:03:15.762195 1951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:15.762274 1951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:15.762359 1951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:15.769763 1951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:15.769857 1951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:15.769989 1951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:15.770044 1951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:15.770109 1951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:15.780445 1951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:36371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:34439,127.0.205.189:38183,127.0.205.188:44785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab
--principal=kudu/127.0.205.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.189:38183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.205.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:15.781808 1951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:15.783249 1951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:15.795972 1957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:15.796033 1961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:15.796061 1965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:15.799456 1951 server_base.cc:1034] running on GCE node
I20260808 03:03:15.800297 1951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:15.801723 1951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:15.802945 1951 hybrid_clock.cc:648] HybridClock initialized: now 1786158195802861 us; error 58 us; skew 500 ppm
I20260808 03:03:15.803242 1951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:15 dist-test-slave-6ntk krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:15.813736 1951 init.cc:377] Logged in from keytab as kudu/127.0.205.189@KRBTEST.COM (short username kudu)
I20260808 03:03:15.815385 1951 webserver.cc:460] Webserver started at http://127.0.205.189:44823/ using document root <none> and password file <none>
I20260808 03:03:15.816169 1951 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:15.816298 1951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:15.816658 1951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:15.819873 1951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/data/instance:
uuid: "67553df065814a36b1654609444a0800"
format_stamp: "Formatted at 2026-08-08 03:03:15 on dist-test-slave-6ntk"
server_key: "3acbe245f3706aec8e97af848382c349"
server_key_iv: "32c5386f490fe3aa01239c309b3d9295"
server_key_version: "encryptionkey@0"
I20260808 03:03:15.820679 1951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/wal/instance:
uuid: "67553df065814a36b1654609444a0800"
format_stamp: "Formatted at 2026-08-08 03:03:15 on dist-test-slave-6ntk"
server_key: "3acbe245f3706aec8e97af848382c349"
server_key_iv: "32c5386f490fe3aa01239c309b3d9295"
server_key_version: "encryptionkey@0"
I20260808 03:03:15.826071 1951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260808 03:03:15.829468 1977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:15.830848 1951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:03:15.831039 1951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/wal
uuid: "67553df065814a36b1654609444a0800"
format_stamp: "Formatted at 2026-08-08 03:03:15 on dist-test-slave-6ntk"
server_key: "3acbe245f3706aec8e97af848382c349"
server_key_iv: "32c5386f490fe3aa01239c309b3d9295"
server_key_version: "encryptionkey@0"
I20260808 03:03:15.831272 1951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:15.857568 1951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:15.862105 1951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:15.862407 1951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:15.882213 1951 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.189:38183
I20260808 03:03:15.882249 2044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.189:38183 every 8 connection(s)
I20260808 03:03:15.883725 1951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
I20260808 03:03:15.884934 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 1951
I20260808 03:03:15.885175 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-1/wal/instance
I20260808 03:03:15.885797 822 external_mini_cluster.cc:1428] Setting key 10e1c86fd95a40c6a4bd85aea9a8e963
I20260808 03:03:15.888875 2045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:15.899401 2045 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:03:15 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.205.188@KRBTEST.COM; defaulting to no policy
I20260808 03:03:15.922580 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.902911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52548 (local address 127.0.205.190:34439)
0808 03:03:15.903490 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:15.903494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:15.904598 (+ 1104us) server_negotiation.cc:407] Connection header received
0808 03:03:15.906679 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:15.906685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:15.906782 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:15.906918 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:15.908658 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.909413 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.911120 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.911566 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.911719 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:15.915578 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:15.915583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:15.915586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:15.915626 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:15.919039 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:15.919935 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:15.919939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:15.919941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:15.920065 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:15.920527 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:15.920532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:15.920534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:15.920940 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:15.921118 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:15.921850 (+ 732us) server_negotiation.cc:299] Negotiation successful
0808 03:03:15.922154 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":341,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.188@KRBTEST.COM" created.
I20260808 03:03:15.923691 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.903603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34439 (local address 127.0.0.1:52548)
0808 03:03:15.904424 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.904473 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:15.906379 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:15.907194 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:15.907204 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:15.907778 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:15.908424 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:15.908444 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.909599 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.909604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:15.910957 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:15.910971 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.911822 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.911829 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:15.912085 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:15.912854 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:15.912919 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:15.915302 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:15.919205 (+ 3903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:15.919216 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:15.919235 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:15.919787 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:15.920183 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:15.920188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:15.920192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:15.920395 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:15.921077 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:15.921087 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:15.921476 (+ 389us) client_negotiation.cc:769] Sending connection context
0808 03:03:15.921757 (+ 281us) client_negotiation.cc:240] Negotiation successful
0808 03:03:15.922341 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":359,"threads_started":1}
I20260808 03:03:15.925668 2045 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:03:15 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:03:15.942490 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.926677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37476)
0808 03:03:15.926839 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.926861 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:15.926969 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:15.928133 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:15.928140 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:15.928602 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:15.928738 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:15.928746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.930210 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.930217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:15.931064 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:15.931075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.931907 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.931913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:15.932084 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:15.932728 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:15.932744 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:15.935287 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:15.939890 (+ 4603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:15.939894 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:15.939897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:15.940270 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:15.940703 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:15.940707 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:15.940709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:15.940849 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:15.941633 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:15.941636 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:15.941759 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 03:03:15.941986 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 03:03:15.942208 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 03:03:15.942929 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.926974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37476 (local address 127.0.205.189:38183)
0808 03:03:15.927720 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:15.927727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:15.927753 (+ 26us) server_negotiation.cc:407] Connection header received
0808 03:03:15.927797 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:15.927803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:15.927985 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:15.928149 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:15.928923 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.930081 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.931203 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:15.931735 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:15.931875 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:15.935548 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:15.935557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:15.935570 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:15.935598 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:15.939661 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:15.940422 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:15.940434 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:15.940440 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:15.940590 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:15.940969 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:15.940976 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:15.940982 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:15.941505 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:15.941731 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:15.942084 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 03:03:15.942387 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":449,"threads_started":1}
I20260808 03:03:15.945501 2045 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:03:15.946682 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.946207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34452)
0808 03:03:15.946410 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.946540 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 03:03:15.946964 1979 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
I20260808 03:03:15.948554 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.948211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34462)
0808 03:03:15.948361 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.948454 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 03:03:15.948817 2045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:44785: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab.
Entry for principal kudu/127.0.205.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab.
I20260808 03:03:15.965969 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.188:44785
--webserver_interface=127.0.205.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab
--principal=kudu/127.0.205.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:36371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:34439,127.0.205.189:38183,127.0.205.188:44785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:03:15.998486 2045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } attempt: 1
I20260808 03:03:15.999919 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:15.999341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34470)
0808 03:03:15.999615 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:15.999724 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260808 03:03:16.001276 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.000763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34474)
0808 03:03:16.001044 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.001144 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260808 03:03:16.001534 2045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:44785: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
I20260808 03:03:16.026051 1939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } attempt: 5
Aug 08 03:03:16 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:03:16.041348 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.026885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37484)
0808 03:03:16.027120 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.027142 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:16.027277 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:16.027676 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:16.027679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:16.028068 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:16.028247 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.028256 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.029817 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.029820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.030758 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.030771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.031962 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.031965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.032110 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.032787 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:16.032805 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:16.035004 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:16.038861 (+ 3857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.038865 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.038868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.039259 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.039657 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.039660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.039662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.039792 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.040414 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:16.040418 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:16.040550 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 03:03:16.040689 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 03:03:16.040941 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 03:03:16.041378 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.027099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.0.205.189:38183)
0808 03:03:16.027318 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.027325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.027349 (+ 24us) server_negotiation.cc:407] Connection header received
0808 03:03:16.027418 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.027424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.027524 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.027642 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.028424 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.029670 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.030934 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.031787 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.031966 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.035233 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.035242 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.035246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.035288 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.038723 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.039386 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.039392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.039395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.039546 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.039920 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.039924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.039927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.040291 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.040446 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.040821 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.041065 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 03:03:16.042675 1939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:03:16.043826 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.043387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34476)
0808 03:03:16.043551 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.043628 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 03:03:16.044788 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.044466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34486)
0808 03:03:16.044611 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.044692 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 03:03:16.045065 1939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:44785: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
I20260808 03:03:16.069779 2045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } attempt: 2
I20260808 03:03:16.071103 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.070519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34488)
0808 03:03:16.070759 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.070899 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 03:03:16.072196 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.071793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34494)
0808 03:03:16.071953 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.072047 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 03:03:16.072465 2045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:44785: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
I20260808 03:03:16.096350 1939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } attempt: 1
I20260808 03:03:16.097743 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.097202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34506)
0808 03:03:16.097459 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.097561 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260808 03:03:16.098865 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.098540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34518)
0808 03:03:16.098682 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.098757 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 03:03:16.099124 1939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:44785: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
I20260808 03:03:16.163743 2045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } attempt: 3
I20260808 03:03:16.165148 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.164654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34522)
0808 03:03:16.164878 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.164978 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 03:03:16.166410 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.165959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34534)
0808 03:03:16.166194 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.166272 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260808 03:03:16.166625 2045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:44785: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
I20260808 03:03:16.167344 1939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } attempt: 2
I20260808 03:03:16.168409 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.167956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34540)
0808 03:03:16.168149 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.168233 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 03:03:16.169370 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.169037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34550)
0808 03:03:16.169188 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.169265 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 03:03:16.169600 1939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:44785: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
I20260808 03:03:16.275823 1939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } attempt: 3
I20260808 03:03:16.277056 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.276523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34566)
0808 03:03:16.276743 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.276860 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 03:03:16.278205 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.277785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34574)
0808 03:03:16.278010 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.278095 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260808 03:03:16.278414 1939 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:44785: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
I20260808 03:03:16.309849 2045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } attempt: 4
I20260808 03:03:16.311139 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.310620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34586)
0808 03:03:16.310872 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.310966 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 03:03:16.312212 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.311876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34598)
0808 03:03:16.312029 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.312112 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 03:03:16.312532 2045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:44785: Network error: Client connection negotiation failed: client connection to 127.0.205.188:44785: connect: Connection refused (error 111)
W20260808 03:03:16.336000 2054 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:03:16.336429 2054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:16.336556 2054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:16.336716 2054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:16.344347 2054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:16.344501 2054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:16.344610 2054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:16.344707 2054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:16.344821 2054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:16.355278 2054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:36371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:34439,127.0.205.189:38183,127.0.205.188:44785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab
--principal=kudu/127.0.205.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.188:44785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.205.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:16.356745 2054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:16.358228 2054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:16.370669 2061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:16.370787 2064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:16.370853 2062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:16.372604 2054 server_base.cc:1034] running on GCE node
I20260808 03:03:16.373466 2054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:16.375092 2054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:16.376363 2054 hybrid_clock.cc:648] HybridClock initialized: now 1786158196376295 us; error 33 us; skew 500 ppm
I20260808 03:03:16.376809 2054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:16 dist-test-slave-6ntk krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:16.380230 2054 init.cc:377] Logged in from keytab as kudu/127.0.205.188@KRBTEST.COM (short username kudu)
I20260808 03:03:16.382000 2054 webserver.cc:460] Webserver started at http://127.0.205.188:33139/ using document root <none> and password file <none>
I20260808 03:03:16.382781 2054 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:16.382951 2054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:16.383349 2054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:16.386682 2054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/data/instance:
uuid: "30a428bdeabf4ce3a21f0ca685d828f5"
format_stamp: "Formatted at 2026-08-08 03:03:16 on dist-test-slave-6ntk"
server_key: "39a8722b549a179eb7a05738b9f318b1"
server_key_iv: "1617556ee59eea508e268e588071a52a"
server_key_version: "encryptionkey@0"
I20260808 03:03:16.387506 2054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/wal/instance:
uuid: "30a428bdeabf4ce3a21f0ca685d828f5"
format_stamp: "Formatted at 2026-08-08 03:03:16 on dist-test-slave-6ntk"
server_key: "39a8722b549a179eb7a05738b9f318b1"
server_key_iv: "1617556ee59eea508e268e588071a52a"
server_key_version: "encryptionkey@0"
I20260808 03:03:16.393281 2054 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 03:03:16.396925 2073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:16.398479 2054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:03:16.398684 2054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/wal
uuid: "30a428bdeabf4ce3a21f0ca685d828f5"
format_stamp: "Formatted at 2026-08-08 03:03:16 on dist-test-slave-6ntk"
server_key: "39a8722b549a179eb7a05738b9f318b1"
server_key_iv: "1617556ee59eea508e268e588071a52a"
server_key_version: "encryptionkey@0"
I20260808 03:03:16.398932 2054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:16.422240 2054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:16.426946 2054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:16.427240 2054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:16.447352 2054 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.188:44785
I20260808 03:03:16.447360 2148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.188:44785 every 8 connection(s)
I20260808 03:03:16.448872 2054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
I20260808 03:03:16.451164 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 2054
I20260808 03:03:16.451373 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786157811617362-822-0/minicluster-data/master-2/wal/instance
I20260808 03:03:16.451988 822 external_mini_cluster.cc:1428] Setting key 138258017eb03db49d8a7d1293d9329b
I20260808 03:03:16.453660 2150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:16.455670 1939 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } attempt: 4
I20260808 03:03:16.464198 2150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:03:16 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
Aug 08 03:03:16 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
I20260808 03:03:16.477479 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.456350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34620)
0808 03:03:16.456551 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.456573 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:16.456673 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:16.458045 (+ 1372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:16.458051 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:16.458613 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:16.458822 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.458837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.461181 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.461187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.462633 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.462654 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.463955 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.463962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.464191 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.465023 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:16.465042 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:16.467399 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:16.473967 (+ 6568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.473974 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.473978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.474400 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.474971 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.474975 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.474979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.475162 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.476394 (+ 1232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:16.476400 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:16.476592 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 03:03:16.476734 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 03:03:16.477074 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 03:03:16.479053 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.456488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.0.205.188:44785)
0808 03:03:16.457090 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.457094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.457112 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:03:16.457181 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.457186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.457618 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.458618 (+ 1000us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.459075 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.461006 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.462809 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.463755 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.464500 (+ 745us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.467728 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.467740 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.467763 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.467824 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.473773 (+ 5949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.474544 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.474552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.474561 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.474801 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.475329 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.475334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.475337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.476220 (+ 883us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.477910 (+ 1690us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.478026 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.478290 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":503,"spinlock_wait_cycles":58496,"thread_start_us":363,"threads_started":1}
I20260808 03:03:16.479089 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.454121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.0.205.188:44785)
0808 03:03:16.454929 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.454941 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.454982 (+ 41us) server_negotiation.cc:407] Connection header received
0808 03:03:16.457156 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.457180 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.457615 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.458038 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.459934 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.461258 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.463029 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.463751 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.463960 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.469613 (+ 5653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.469623 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.469631 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.469684 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.474313 (+ 4629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.475331 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.475335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.475337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.475543 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.476006 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.476011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.476015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.476427 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.476841 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.477218 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.477707 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":606,"thread_start_us":382,"threads_started":1}
Aug 08 03:03:16 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:03:16.487787 1939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27: Bootstrap starting.
I20260808 03:03:16.491933 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.467684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52560 (local address 127.0.205.190:34439)
0808 03:03:16.471525 (+ 3841us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.471532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.472342 (+ 810us) server_negotiation.cc:407] Connection header received
0808 03:03:16.472664 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.472673 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.472757 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.472871 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.474691 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.475900 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.478558 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.479286 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.479417 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.484364 (+ 4947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.484371 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.484375 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.484416 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.488755 (+ 4339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.489658 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.489664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.489668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.489811 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.490288 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.490300 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.490302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.490600 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.490789 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.491252 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.491479 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3665,"thread_start_us":3168,"threads_started":1}
I20260808 03:03:16.491933 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.467634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:34439 (local address 127.0.0.1:52560)
0808 03:03:16.472173 (+ 4539us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.472236 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:16.472493 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:16.472959 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:16.472968 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:16.474129 (+ 1161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:16.474539 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.474552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.476121 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.476128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.478380 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.478395 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.479660 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.479665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.479923 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.481094 (+ 1171us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:16.481118 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:16.484091 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:16.488943 (+ 4852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.488961 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.488968 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.489518 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.489989 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.489997 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.490001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.490155 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.490742 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:16.490750 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:16.490943 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 03:03:16.491139 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 03:03:16.491476 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4225,"mutex_wait_us":21,"thread_start_us":4041,"threads_started":1}
I20260808 03:03:16.492494 1939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:16.494278 2150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:03:16.494426 1939 log.cc:824] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27: Log is configured to *not* fsync() on all Append() calls
Aug 08 03:03:16 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:03:16.500254 1939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27: No bootstrap required, opened a new log
Aug 08 03:03:16 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:03:16.507895 2159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.482506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.0.205.190:34439)
0808 03:03:16.488703 (+ 6197us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.488707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.488728 (+ 21us) server_negotiation.cc:407] Connection header received
0808 03:03:16.488775 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.488778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.488863 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.488982 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.490827 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.492124 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.493571 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.494207 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.494345 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.501145 (+ 6800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.501153 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.501158 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.501196 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.504806 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.505543 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.505554 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.505559 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.505726 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.506184 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.506189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.506192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.506481 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.506760 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.507118 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.507356 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6079,"thread_start_us":5828,"threads_started":1}
I20260808 03:03:16.510293 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.495238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:38183 (local address 127.0.0.1:37500)
0808 03:03:16.495582 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.495606 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:16.495750 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:16.496095 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:16.496102 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:16.496592 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:16.496738 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.496746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.498160 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.498165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.499024 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.499035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.499759 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.499764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.499956 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.500544 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:16.500561 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:16.503223 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:16.507414 (+ 4191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.507420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.507424 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.507974 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.508439 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.508443 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.508447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.508602 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.509240 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:16.509246 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:16.509424 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 03:03:16.509584 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 03:03:16.509928 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260808 03:03:16.510450 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.495123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.0.205.189:38183)
0808 03:03:16.495293 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.495297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.495685 (+ 388us) server_negotiation.cc:407] Connection header received
0808 03:03:16.495869 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.495873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.495937 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.496051 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.496905 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.498010 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.499170 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.499599 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.499739 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.503491 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.503525 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.503530 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.503576 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.507244 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.508125 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.508131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.508136 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.508306 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.508727 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.508733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.508736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.509086 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.509249 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.509702 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.509965 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 03:03:16.511880 2150 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:03:16.512555 1939 raft_consensus.cc:348] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } }
I20260808 03:03:16.512995 1939 raft_consensus.cc:729] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeaa82f8aa2a4949a7a88c06c9210c27, State: Initialized, Role: FOLLOWER
I20260808 03:03:16.513619 1939 consensus_queue.cc:260] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } }
I20260808 03:03:16.514627 2163 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } } }
I20260808 03:03:16.515136 2163 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:16.516073 1939 sys_catalog.cc:564] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 03:03:16 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
Aug 08 03:03:16 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:03:16.530092 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.512772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.0.205.188:44785)
0808 03:03:16.512998 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.513007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.513284 (+ 277us) server_negotiation.cc:407] Connection header received
0808 03:03:16.513322 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.513326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.513387 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.513480 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.514581 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.515640 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.517229 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.517658 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.517791 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.522210 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.522216 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.522221 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.522256 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.526345 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.527140 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.527149 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.527156 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.527362 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.527809 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.527814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.527817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.528294 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.528494 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.529125 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.529657 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":276,"server-negotiator.queue_time_us":64}
I20260808 03:03:16.530169 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.512672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34630)
0808 03:03:16.513054 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.513076 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:16.513188 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:16.513544 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:16.513553 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:16.514230 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:16.514431 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.514443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.515807 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.515813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.517077 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.517089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.517804 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.517809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.518040 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.518747 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:16.518780 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:16.522011 (+ 3231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:16.526485 (+ 4474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.526492 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.526497 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.526985 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.527507 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.527512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.527516 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.527688 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.528454 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:16.528467 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:16.528748 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 03:03:16.528995 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 03:03:16.529492 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":1}
I20260808 03:03:16.533749 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.510227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.0.205.189:38183)
0808 03:03:16.510878 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.510887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.514772 (+ 3885us) server_negotiation.cc:407] Connection header received
0808 03:03:16.514829 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.514837 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.514919 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.515006 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.516824 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.518220 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.519534 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.519964 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.520105 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.525822 (+ 5717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.525831 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.525837 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.525943 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.529573 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.530574 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.530578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.530582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.530742 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.531293 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.531299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.531304 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.531671 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.532950 (+ 1279us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.533114 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.533362 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":4014,"threads_started":1}
I20260808 03:03:16.539996 2150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5: Bootstrap starting.
W20260808 03:03:16.540017 2180 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:03:16.540150 2180 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:03:16.541923 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 03:03:16.544117 2150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:16.545684 2150 log.cc:824] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5: Log is configured to *not* fsync() on all Append() calls
I20260808 03:03:16.545835 1900 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52576
I20260808 03:03:16.546440 1900 raft_consensus.cc:482] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 03:03:16.546644 1900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:16.539574 1843 master_runner.cc:186] Error getting master registration for 127.0.205.189:38183: OK, instance_id { permanent_uuid: "67553df065814a36b1654609444a0800" instance_seqno: 1786158195876394 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260808 03:03:16.548487 1869 tablet.cc:2369] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:03:16.548691 1843 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:03:16.550371 1900 raft_consensus.cc:504] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } }
I20260808 03:03:16.551504 2150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5: No bootstrap required, opened a new log
I20260808 03:03:16.552932 1900 leader_election.cc:288] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [CANDIDATE]: Term 1 election: Requested vote from peers 67553df065814a36b1654609444a0800 (127.0.205.189:38183), 30a428bdeabf4ce3a21f0ca685d828f5 (127.0.205.188:44785)
I20260808 03:03:16.563310 2150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } }
I20260808 03:03:16.563774 2150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a428bdeabf4ce3a21f0ca685d828f5, State: Initialized, Role: FOLLOWER
I20260808 03:03:16.564347 2150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } }
I20260808 03:03:16.571326 2198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } } }
I20260808 03:03:16.571743 2198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:16.573629 2150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:16.573716 2115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "30a428bdeabf4ce3a21f0ca685d828f5"
I20260808 03:03:16.574302 2115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:16.580102 2115 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeaa82f8aa2a4949a7a88c06c9210c27 in term 1.
I20260808 03:03:16.581851 1865 leader_election.cc:302] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a428bdeabf4ce3a21f0ca685d828f5, aeaa82f8aa2a4949a7a88c06c9210c27; no voters:
I20260808 03:03:16.582577 2163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:03:16.595067 2163 raft_consensus.cc:686] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [term 1 LEADER]: Becoming Leader. State: Replica: aeaa82f8aa2a4949a7a88c06c9210c27, State: Running, Role: LEADER
I20260808 03:03:16.597657 2163 consensus_queue.cc:237] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } }
I20260808 03:03:16.608860 2159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.562456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52582 (local address 127.0.205.190:34439)
0808 03:03:16.562660 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.562666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.574591 (+ 11925us) server_negotiation.cc:407] Connection header received
0808 03:03:16.574846 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.574852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.574989 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.575121 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.584176 (+ 9055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.585290 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.587805 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.588521 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.588622 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.595044 (+ 6422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.595052 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.595056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.595104 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.600371 (+ 5267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.603840 (+ 3469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.603847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.603851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.604033 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.605253 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.605258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.605262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.605672 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.605981 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.608113 (+ 2132us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.608400 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 03:03:16.609266 2207 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aeaa82f8aa2a4949a7a88c06c9210c27. Latest consensus state: current_term: 1 leader_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } } }
I20260808 03:03:16.610366 2207 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:16.610952 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.568094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.0.205.189:38183)
0808 03:03:16.568568 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.568577 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.576189 (+ 7612us) server_negotiation.cc:407] Connection header received
0808 03:03:16.577200 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.577206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.577303 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.577379 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.582007 (+ 4628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.583263 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.587301 (+ 4038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.588035 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.588145 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.595110 (+ 6965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.595116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.595120 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.595170 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.602250 (+ 7080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.604582 (+ 2332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.604588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.604592 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.604790 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.606498 (+ 1708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.606503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.606506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.606905 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.607475 (+ 570us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.610264 (+ 2789us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.610542 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260808 03:03:16.611756 2045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } attempt: 5
I20260808 03:03:16.618538 2213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:03:16.620141 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.576859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34640 (local address 127.0.205.188:44785)
0808 03:03:16.581236 (+ 4377us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.581246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.590972 (+ 9726us) server_negotiation.cc:407] Connection header received
0808 03:03:16.591061 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.591068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.591205 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.591327 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.601771 (+ 10444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.602931 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.605234 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.606009 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.606112 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.610170 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.610176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.610181 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.610223 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.615501 (+ 5278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.616592 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.616598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.616601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.616802 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.617399 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.617405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.617409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.617910 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.618219 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.619435 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.619712 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3545}
Aug 08 03:03:16 dist-test-slave-6ntk krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
I20260808 03:03:16.622457 2213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:03:16.664996 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.612597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:44785 (local address 127.0.0.1:34650)
0808 03:03:16.612860 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:16.612878 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:16.612986 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:16.613969 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:16.613976 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:16.614465 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:16.614710 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.614723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.616267 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.616273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.617755 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:16.617773 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.618931 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.618939 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:16.619217 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.619919 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:16.619947 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:16.623020 (+ 3073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:16.651863 (+ 28843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.651877 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.651886 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.652862 (+ 976us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.654501 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:16.654513 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:16.654521 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:16.654833 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:16.663073 (+ 8240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:16.663086 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:16.663391 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 03:03:16.663636 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 03:03:16.664203 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260808 03:03:16.675508 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:16.613160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.0.205.188:44785)
0808 03:03:16.613418 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:16.613425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:16.613445 (+ 20us) server_negotiation.cc:407] Connection header received
0808 03:03:16.613503 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:16.613512 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:16.613598 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:16.613713 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:16.614994 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.616093 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.617957 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:16.618607 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:16.618713 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:16.623325 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:16.623332 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:16.623337 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:16.623376 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:16.651388 (+ 28012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.653346 (+ 1958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.653359 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.653366 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.654020 (+ 654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:16.655344 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:16.655392 (+ 48us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:16.655402 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:16.662696 (+ 7294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:16.666528 (+ 3832us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:16.667051 (+ 523us) server_negotiation.cc:299] Negotiation successful
0808 03:03:16.673796 (+ 6745us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260808 03:03:16.679560 2081 tablet.cc:2369] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:03:16.696295 2045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800: Bootstrap starting.
I20260808 03:03:16.707653 2045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:16.713053 2115 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [term 1 FOLLOWER]: Refusing update from remote peer aeaa82f8aa2a4949a7a88c06c9210c27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:03:16.714103 2045 log.cc:824] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800: Log is configured to *not* fsync() on all Append() calls
I20260808 03:03:16.714875 2207 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:03:16.725503 2045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800: No bootstrap required, opened a new log
I20260808 03:03:16.736063 2198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aeaa82f8aa2a4949a7a88c06c9210c27. Latest consensus state: current_term: 1 leader_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } } }
I20260808 03:03:16.736553 2198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:16.744331 2045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } }
I20260808 03:03:16.745076 2045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67553df065814a36b1654609444a0800, State: Initialized, Role: FOLLOWER
I20260808 03:03:16.745961 2045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } }
I20260808 03:03:16.768481 2225 mvcc.cc:205] Tried to move back new op lower bound from 7316103973721186304 to 7316103973271146496. Current Snapshot: MvccSnapshot[applied={T|T < 7316103973721186304}]
I20260808 03:03:16.785142 2198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } } }
I20260808 03:03:16.785480 2198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:16.789702 2207 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } } }
I20260808 03:03:16.790162 2207 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:16.792445 2213 catalog_manager.cc:1347] Generated new cluster ID: 75864dd999e1444da4697e0b7f6954af
I20260808 03:03:16.792590 2213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 03:03:16.746078 2054 master_runner.cc:186] Error getting master registration for 127.0.205.189:38183: OK, instance_id { permanent_uuid: "67553df065814a36b1654609444a0800" instance_seqno: 1786158195876394 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:03:16.793238 2054 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:03:16.797461 2045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:16.798014 2014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:16.802208 2014 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [term 1 FOLLOWER]: Refusing update from remote peer aeaa82f8aa2a4949a7a88c06c9210c27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:03:16.802021 2015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "67553df065814a36b1654609444a0800"
I20260808 03:03:16.803395 2163 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 03:03:16.808240 2227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:03:16.811970 2227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:03:16.797319 2229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } } }
I20260808 03:03:16.816107 2015 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeaa82f8aa2a4949a7a88c06c9210c27 in term 1.
I20260808 03:03:16.816735 2229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:16.826254 2213 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:03:16.828686 2213 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:03:16.845726 2232 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } } }
I20260808 03:03:16.846253 2232 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:16.847072 2229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aeaa82f8aa2a4949a7a88c06c9210c27. Latest consensus state: current_term: 1 leader_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } } }
I20260808 03:03:16.847601 2229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:16.850342 2235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeaa82f8aa2a4949a7a88c06c9210c27" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 34439 } } peers { permanent_uuid: "67553df065814a36b1654609444a0800" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 38183 } } peers { permanent_uuid: "30a428bdeabf4ce3a21f0ca685d828f5" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 44785 } } }
I20260808 03:03:16.850828 2235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:16.866066 2213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aeaa82f8aa2a4949a7a88c06c9210c27: Generated new TSK 0
I20260808 03:03:16.867295 2213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 03:03:16.969224 2254 catalog_manager.cc:1259] Loaded cluster ID: 75864dd999e1444da4697e0b7f6954af
I20260808 03:03:16.969430 2254 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800: loading cluster ID for follower catalog manager: success
I20260808 03:03:16.973493 2254 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800: acquiring CA information for follower catalog manager: success
I20260808 03:03:16.976533 2254 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 67553df065814a36b1654609444a0800: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 03:03:17.006181 822 mini_kdc.cc:348] Time spent kdestroy: real 0.127s user 0.000s sys 0.032s
W20260808 03:03:17.057415 2159 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.040500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.0.205.190:34439)
0808 03:03:17.040720 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.040729 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.054722 (+ 13993us) server_negotiation.cc:407] Connection header received
0808 03:03:17.054977 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.054983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.055076 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.055205 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.057079 (+ 1874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52598: BlockingRecv error: recv got EOF from 127.0.0.1:52598 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 03:03:17.061129 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.058827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.0.205.188:44785)
0808 03:03:17.059049 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.059056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.059191 (+ 135us) server_negotiation.cc:407] Connection header received
0808 03:03:17.059432 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.059441 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.059540 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.059629 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.060842 (+ 1213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34662: BlockingRecv error: recv got EOF from 127.0.0.1:34662 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 03:03:17.062533 2050 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.049333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37530 (local address 127.0.205.189:38183)
0808 03:03:17.049636 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.049646 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.057686 (+ 8040us) server_negotiation.cc:407] Connection header received
0808 03:03:17.057788 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.057796 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.057941 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.058064 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.062277 (+ 4213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37530: BlockingRecv error: recv got EOF from 127.0.0.1:37530 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 03:03:17.110515 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.095047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37532 (local address 127.0.205.189:38183)
0808 03:03:17.095266 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.095274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.101929 (+ 6655us) server_negotiation.cc:407] Connection header received
0808 03:03:17.102470 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.102478 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.102737 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.102947 (+ 210us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.103621 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.104958 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.107177 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.107997 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.108155 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:17.108858 (+ 703us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:17.109644 (+ 786us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:17.109821 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:17.109990 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 03:03:17.110101 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260808 03:03:17.170024 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.154188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34668 (local address 127.0.205.188:44785)
0808 03:03:17.154400 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.154406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.167655 (+ 13249us) server_negotiation.cc:407] Connection header received
0808 03:03:17.168233 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.168240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.168394 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.168479 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.169701 (+ 1222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34668: BlockingRecv error: recv got EOF from 127.0.0.1:34668 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 03:03:17.221639 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.207236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37548 (local address 127.0.205.189:38183)
0808 03:03:17.207463 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.207470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.214478 (+ 7008us) server_negotiation.cc:407] Connection header received
0808 03:03:17.215051 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.215058 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.215154 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.215226 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.215962 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.217113 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.219306 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.220004 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.220110 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:17.220746 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:17.220909 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:17.220979 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:17.221215 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 03:03:17.221321 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 03:03:17.269251 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.256074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.0.205.188:44785)
0808 03:03:17.256269 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.256275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.267069 (+ 10794us) server_negotiation.cc:407] Connection header received
0808 03:03:17.267615 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.267621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.267714 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.267803 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.268972 (+ 1169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34674: BlockingRecv error: recv got EOF from 127.0.0.1:34674 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:03:17.293931 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.286331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37558 (local address 127.0.205.189:38183)
0808 03:03:17.286561 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.286567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.287177 (+ 610us) server_negotiation.cc:407] Connection header received
0808 03:03:17.287406 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.287413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.287518 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.287629 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.288198 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.289417 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.291570 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.292284 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.292433 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:17.292957 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:17.293121 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:17.293217 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:17.293413 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 03:03:17.293502 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260808 03:03:17.310489 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.308164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.0.205.188:44785)
0808 03:03:17.308378 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.308385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.308735 (+ 350us) server_negotiation.cc:407] Connection header received
0808 03:03:17.308994 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.309000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.309099 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.309209 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.310182 (+ 973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34680: BlockingRecv error: recv got EOF from 127.0.0.1:34680 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:03:17.354586 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.346389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.0.205.189:38183)
0808 03:03:17.346666 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.346683 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.347583 (+ 900us) server_negotiation.cc:407] Connection header received
0808 03:03:17.347827 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.347834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.347957 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.348088 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.348627 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.349831 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.352173 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.352892 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.353035 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:17.353611 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:17.353760 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:17.353906 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:17.354080 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 03:03:17.354178 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
W20260808 03:03:17.365389 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.363452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.0.205.188:44785)
0808 03:03:17.363708 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.363715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.363747 (+ 32us) server_negotiation.cc:407] Connection header received
0808 03:03:17.363833 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.363839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.363942 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.364064 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.365038 (+ 974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34682: BlockingRecv error: recv got EOF from 127.0.0.1:34682 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
I20260808 03:03:17.392866 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.384989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37578 (local address 127.0.205.189:38183)
0808 03:03:17.385191 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.385200 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.385836 (+ 636us) server_negotiation.cc:407] Connection header received
0808 03:03:17.386126 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.386138 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.386230 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.386344 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.386889 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.388122 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.390493 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.391237 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.391368 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:17.391934 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:17.392118 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:17.392207 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:17.392407 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 03:03:17.392524 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 03:03:17.403307 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.400419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.0.205.188:44785)
0808 03:03:17.400630 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.400638 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.401213 (+ 575us) server_negotiation.cc:407] Connection header received
0808 03:03:17.401548 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.401555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.401688 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.401770 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.403014 (+ 1244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34690: BlockingRecv error: recv got EOF from 127.0.0.1:34690 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 03:03:17.435681 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.427788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37594 (local address 127.0.205.189:38183)
0808 03:03:17.428034 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.428044 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.428709 (+ 665us) server_negotiation.cc:407] Connection header received
0808 03:03:17.428964 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.428970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.429067 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.429166 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.429841 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.431183 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.433484 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.434071 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.434174 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:17.434821 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:17.434964 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:17.435035 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:17.435294 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 03:03:17.435382 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260808 03:03:17.444653 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.442071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.0.205.188:44785)
0808 03:03:17.442323 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.442330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.442977 (+ 647us) server_negotiation.cc:407] Connection header received
0808 03:03:17.443192 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.443196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.443281 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.443366 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.444427 (+ 1061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34694: BlockingRecv error: recv got EOF from 127.0.0.1:34694 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 03:03:17.488001 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.481105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37598 (local address 127.0.205.189:38183)
0808 03:03:17.481363 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.481370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.481798 (+ 428us) server_negotiation.cc:407] Connection header received
0808 03:03:17.481979 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.481987 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.482128 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.482263 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.482667 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.484009 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.485518 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.486391 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.486553 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:17.486993 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:17.487170 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:17.487283 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:17.487437 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 03:03:17.487541 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260808 03:03:17.496791 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.494567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34706 (local address 127.0.205.188:44785)
0808 03:03:17.494788 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.494794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.495166 (+ 372us) server_negotiation.cc:407] Connection header received
0808 03:03:17.495384 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.495389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.495469 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.495554 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.496559 (+ 1005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34706: BlockingRecv error: recv got EOF from 127.0.0.1:34706 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 03:03:17.570575 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.565141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37612 (local address 127.0.205.189:38183)
0808 03:03:17.565398 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.565408 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.566058 (+ 650us) server_negotiation.cc:407] Connection header received
0808 03:03:17.566294 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.566298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.566370 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.566469 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.566957 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.567673 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.568975 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.569389 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.569491 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:17.569859 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:17.570050 (+ 191us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:17.570122 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:17.570292 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 03:03:17.570361 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 03:03:17.578474 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.576589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34714 (local address 127.0.205.188:44785)
0808 03:03:17.576759 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.576763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.577147 (+ 384us) server_negotiation.cc:407] Connection header received
0808 03:03:17.577303 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.577307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.577380 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.577469 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.578290 (+ 821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34714: BlockingRecv error: recv got EOF from 127.0.0.1:34714 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 03:03:17.717123 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.711827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37618 (local address 127.0.205.189:38183)
0808 03:03:17.712106 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.712110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.712490 (+ 380us) server_negotiation.cc:407] Connection header received
0808 03:03:17.712674 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.712678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.712749 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.712841 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.713308 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.714053 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.715434 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.715896 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.716011 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:17.716400 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:17.716507 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:17.716598 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:17.716753 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 03:03:17.716822 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
W20260808 03:03:17.724720 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:17.722817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34716 (local address 127.0.205.188:44785)
0808 03:03:17.723006 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.723011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.723394 (+ 383us) server_negotiation.cc:407] Connection header received
0808 03:03:17.723555 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.723559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.723632 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.723716 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:17.724508 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34716: BlockingRecv error: recv got EOF from 127.0.0.1:34716 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 03:03:17.819267 2227 catalog_manager.cc:1259] Loaded cluster ID: 75864dd999e1444da4697e0b7f6954af
I20260808 03:03:17.819429 2227 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5: loading cluster ID for follower catalog manager: success
I20260808 03:03:17.822049 2227 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5: acquiring CA information for follower catalog manager: success
I20260808 03:03:17.823877 2227 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 30a428bdeabf4ce3a21f0ca685d828f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 03:03:17.989663 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.984365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37620 (local address 127.0.205.189:38183)
0808 03:03:17.984607 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.984612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.984782 (+ 170us) server_negotiation.cc:407] Connection header received
0808 03:03:17.984951 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.984955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.985037 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.985142 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.985572 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.986545 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.987908 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.988334 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.988486 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:17.988918 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:17.989078 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:17.989155 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:17.989369 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 03:03:17.989454 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260808 03:03:18.002065 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:17.996007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34724 (local address 127.0.205.188:44785)
0808 03:03:17.996193 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:17.996198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:17.996529 (+ 331us) server_negotiation.cc:407] Connection header received
0808 03:03:17.996687 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:17.996691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:17.996866 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:17.997004 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:17.997531 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:17.998360 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:17.999771 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:18.000190 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:18.000290 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:18.000820 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:18.001429 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:18.001544 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:18.001667 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 03:03:18.001737 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 03:03:18.018325 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:18.008477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37624 (local address 127.0.205.189:38183)
0808 03:03:18.008666 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:18.008673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:18.009401 (+ 728us) server_negotiation.cc:407] Connection header received
0808 03:03:18.010741 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:18.010750 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:18.010864 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:18.010964 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:18.011536 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:18.012563 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:18.014684 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:18.015347 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:18.015497 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:18.016643 (+ 1146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:18.016823 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:18.016915 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:18.017823 (+ 908us) server_negotiation.cc:299] Negotiation successful
0808 03:03:18.017949 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 03:03:18.018469 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:18.008617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52606 (local address 127.0.205.190:34439)
0808 03:03:18.010693 (+ 2076us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:18.010703 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:18.010730 (+ 27us) server_negotiation.cc:407] Connection header received
0808 03:03:18.010882 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:18.010898 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:18.011118 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:18.011279 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:18.012023 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:18.013085 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:18.015185 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:18.015796 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:18.016495 (+ 699us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:18.016692 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:18.017768 (+ 1076us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:18.017915 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:18.018087 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 03:03:18.018156 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1903,"thread_start_us":1777,"threads_started":1}
I20260808 03:03:18.018803 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:18.009431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.0.205.188:44785)
0808 03:03:18.009603 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:18.009608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:18.011917 (+ 2309us) server_negotiation.cc:407] Connection header received
0808 03:03:18.012090 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:18.012094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:18.012179 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:18.012280 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:18.012852 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:18.013679 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:18.015658 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:18.016311 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:18.016465 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:18.016953 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:18.017065 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:18.017156 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:18.017301 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 03:03:18.017383 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 03:03:18.024205 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1843
I20260808 03:03:18.025461 1931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:03:18.141098 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1843
I20260808 03:03:18.170974 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 1951
I20260808 03:03:18.172082 2039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:03:18.287983 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 1951
I20260808 03:03:18.320058 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2054
I20260808 03:03:18.321286 2140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:03:18.430902 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 2054
2026-08-08T03:03:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5318 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 03:03:18 dist-test-slave-6ntk krb5kdc[2447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 03:03:18 dist-test-slave-6ntk krb5kdc[2447](info): set up 2 sockets
Aug 08 03:03:18 dist-test-slave-6ntk krb5kdc[2447](info): commencing operation
krb5kdc: starting...
W20260808 03:03:19.791559 822 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.002s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/test-user.keytab.
Aug 08 03:03:20 dist-test-slave-6ntk krb5kdc[2447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T03:03:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T03:03:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.205.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
Entry for principal kudu/127.0.205.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab.
I20260808 03:03:20.161388 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:42153
--webserver_interface=127.0.205.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:45259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:42153,127.0.205.189:39043,127.0.205.188:42707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 03:03:20.529126 2463 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:03:20.529569 2463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:20.529651 2463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:20.529735 2463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:20.537259 2463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:20.537369 2463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:20.537431 2463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:20.537482 2463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:20.537540 2463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:20.548589 2463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:45259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:42153,127.0.205.189:39043,127.0.205.188:42707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.190.keytab
--principal=kudu/127.0.205.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.190:42153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.205.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:20.550495 2463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:20.551937 2463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:20.564993 2469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:20.564957 2470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:20.564893 2472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:20.566025 2463 server_base.cc:1034] running on GCE node
I20260808 03:03:20.566916 2463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:20.568439 2463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:20.569702 2463 hybrid_clock.cc:648] HybridClock initialized: now 1786158200569632 us; error 43 us; skew 500 ppm
I20260808 03:03:20.570127 2463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:20 dist-test-slave-6ntk krb5kdc[2447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:20.573509 2463 init.cc:377] Logged in from keytab as kudu/127.0.205.190@KRBTEST.COM (short username kudu)
I20260808 03:03:20.575151 2463 webserver.cc:460] Webserver started at http://127.0.205.190:37829/ using document root <none> and password file <none>
I20260808 03:03:20.575884 2463 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:20.575977 2463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:20.576261 2463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:20.579530 2463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/data/instance:
uuid: "a62d96eeda34413da9abda08d956cd44"
format_stamp: "Formatted at 2026-08-08 03:03:20 on dist-test-slave-6ntk"
server_key: "bb3853b0963eab28a057b0d373970452"
server_key_iv: "7850100582c01b5ed362788dad878b5b"
server_key_version: "encryptionkey@0"
I20260808 03:03:20.580423 2463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/wal/instance:
uuid: "a62d96eeda34413da9abda08d956cd44"
format_stamp: "Formatted at 2026-08-08 03:03:20 on dist-test-slave-6ntk"
server_key: "bb3853b0963eab28a057b0d373970452"
server_key_iv: "7850100582c01b5ed362788dad878b5b"
server_key_version: "encryptionkey@0"
I20260808 03:03:20.586318 2463 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 03:03:20.590197 2480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:20.591777 2463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 03:03:20.591998 2463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/wal
uuid: "a62d96eeda34413da9abda08d956cd44"
format_stamp: "Formatted at 2026-08-08 03:03:20 on dist-test-slave-6ntk"
server_key: "bb3853b0963eab28a057b0d373970452"
server_key_iv: "7850100582c01b5ed362788dad878b5b"
server_key_version: "encryptionkey@0"
I20260808 03:03:20.592348 2463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:20.629359 2463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:20.634308 2463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:20.634747 2463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:20.655040 2463 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.190:42153
I20260808 03:03:20.655061 2570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.190:42153 every 8 connection(s)
I20260808 03:03:20.656548 2463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/data/info.pb
I20260808 03:03:20.658562 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 2463
I20260808 03:03:20.658823 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-0/wal/instance
I20260808 03:03:20.659313 822 external_mini_cluster.cc:1428] Setting key 9112799abc1481028a7d9af959bd2e78
I20260808 03:03:20.660686 2571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:20.667748 2571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:03:20 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.205.189@KRBTEST.COM; defaulting to no policy
I20260808 03:03:20.692679 2573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:20.671458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.0.205.190:42153)
0808 03:03:20.672428 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:20.672447 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:20.672922 (+ 475us) server_negotiation.cc:407] Connection header received
0808 03:03:20.675767 (+ 2845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:20.675773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:20.676033 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:20.676265 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:20.677636 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:20.678926 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:20.680412 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:20.680980 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:20.681202 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:20.685063 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:20.685071 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:20.685084 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:20.685113 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:20.688088 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:20.688822 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:20.688835 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:20.688837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:20.688989 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:20.689405 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:20.689410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:20.689415 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:20.690010 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:20.690236 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:20.690768 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 03:03:20.691157 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":668,"thread_start_us":433,"threads_started":1}
I20260808 03:03:20.692767 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:20.671473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:42153 (local address 127.0.0.1:49316)
0808 03:03:20.672731 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:20.672776 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:20.675315 (+ 2539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:20.676285 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:20.676292 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:20.677059 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:20.677412 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:20.677430 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:20.679149 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:20.679154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:20.680253 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:20.680265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:20.681240 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:20.681248 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:20.681419 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:20.682214 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:20.682238 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:20.684817 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:20.688222 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:20.688236 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:20.688249 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:20.688680 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:20.689109 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:20.689115 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:20.689118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:20.689285 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:20.690149 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:20.690162 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:20.690389 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 03:03:20.690656 (+ 267us) client_negotiation.cc:240] Negotiation successful
0808 03:03:20.691284 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"mutex_wait_us":142,"spinlock_wait_cycles":51456,"thread_start_us":794,"threads_started":1}
I20260808 03:03:20.695978 2571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.189@KRBTEST.COM" created.
I20260808 03:03:20.697260 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:20.696781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48372)
0808 03:03:20.697003 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:20.697136 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 03:03:20.697517 2488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
I20260808 03:03:20.699338 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:20.698826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48384)
0808 03:03:20.699009 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:20.699105 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 03:03:20.699549 2571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:39043: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab.
Entry for principal kudu/127.0.205.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab.
I20260808 03:03:20.737437 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.189:39043
--webserver_interface=127.0.205.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab
--principal=kudu/127.0.205.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:45259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:42153,127.0.205.189:39043,127.0.205.188:42707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:03:20.748980 2571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } attempt: 1
I20260808 03:03:20.750582 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:20.749842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48394)
0808 03:03:20.750196 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:20.750375 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260808 03:03:20.751808 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:20.751303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48400)
0808 03:03:20.751493 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:20.751598 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 03:03:20.752070 2571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:39043: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
I20260808 03:03:20.820399 2571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } attempt: 2
I20260808 03:03:20.821719 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:20.821169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48410)
0808 03:03:20.821467 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:20.821570 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 03:03:20.823081 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:20.822657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48426)
0808 03:03:20.822843 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:20.822939 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260808 03:03:20.823406 2571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:39043: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
I20260808 03:03:20.914744 2571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } attempt: 3
I20260808 03:03:20.916184 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:20.915684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48432)
0808 03:03:20.915946 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:20.916043 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260808 03:03:20.917559 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:20.917096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48434)
0808 03:03:20.917293 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:20.917401 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260808 03:03:20.917964 2571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:39043: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
I20260808 03:03:21.061298 2571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } attempt: 4
I20260808 03:03:21.062752 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.062134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48438)
0808 03:03:21.062484 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.062585 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216}
I20260808 03:03:21.063982 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.063607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48454)
0808 03:03:21.063764 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.063852 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 03:03:21.064257 2571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.189:39043: Network error: Client connection negotiation failed: client connection to 127.0.205.189:39043: connect: Connection refused (error 111)
W20260808 03:03:21.096901 2580 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:03:21.097307 2580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:21.097435 2580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:21.097554 2580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:21.105633 2580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:21.105738 2580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:21.105799 2580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:21.105854 2580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:21.106014 2580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:21.116567 2580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:45259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:42153,127.0.205.189:39043,127.0.205.188:42707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.189.keytab
--principal=kudu/127.0.205.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.189:39043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.205.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:21.118002 2580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:21.119422 2580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:21.131963 2594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:21.131995 2589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:21.132019 2591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:21.132674 2580 server_base.cc:1034] running on GCE node
I20260808 03:03:21.133570 2580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:21.135115 2580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:21.136377 2580 hybrid_clock.cc:648] HybridClock initialized: now 1786158201136307 us; error 41 us; skew 500 ppm
I20260808 03:03:21.136734 2580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:21.144568 2580 init.cc:377] Logged in from keytab as kudu/127.0.205.189@KRBTEST.COM (short username kudu)
I20260808 03:03:21.146298 2580 webserver.cc:460] Webserver started at http://127.0.205.189:43431/ using document root <none> and password file <none>
I20260808 03:03:21.147074 2580 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:21.147221 2580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:21.147570 2580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:21.150832 2580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/data/instance:
uuid: "895a376384614816875777d20ee7ba58"
format_stamp: "Formatted at 2026-08-08 03:03:21 on dist-test-slave-6ntk"
server_key: "c3a8f87493babad950961f4b3cdb7f36"
server_key_iv: "9c15366cafe26557ca69dd1dbed413b6"
server_key_version: "encryptionkey@0"
I20260808 03:03:21.151687 2580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/wal/instance:
uuid: "895a376384614816875777d20ee7ba58"
format_stamp: "Formatted at 2026-08-08 03:03:21 on dist-test-slave-6ntk"
server_key: "c3a8f87493babad950961f4b3cdb7f36"
server_key_iv: "9c15366cafe26557ca69dd1dbed413b6"
server_key_version: "encryptionkey@0"
I20260808 03:03:21.157186 2580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 03:03:21.160832 2605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:21.162252 2580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:03:21.162452 2580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/wal
uuid: "895a376384614816875777d20ee7ba58"
format_stamp: "Formatted at 2026-08-08 03:03:21 on dist-test-slave-6ntk"
server_key: "c3a8f87493babad950961f4b3cdb7f36"
server_key_iv: "9c15366cafe26557ca69dd1dbed413b6"
server_key_version: "encryptionkey@0"
I20260808 03:03:21.162688 2580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:21.188017 2580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:21.192343 2580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:21.192655 2580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:21.212488 2580 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.189:39043
I20260808 03:03:21.212512 2685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.189:39043 every 8 connection(s)
I20260808 03:03:21.214011 2580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/data/info.pb
I20260808 03:03:21.214628 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 2580
I20260808 03:03:21.214887 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-1/wal/instance
I20260808 03:03:21.215502 822 external_mini_cluster.cc:1428] Setting key e982d25eb99090f37abc356116f1551c
I20260808 03:03:21.218479 2686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:21.225569 2686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.205.188@KRBTEST.COM; defaulting to no policy
I20260808 03:03:21.249020 2688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.228524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.0.205.190:42153)
0808 03:03:21.232017 (+ 3493us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.232024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.232049 (+ 25us) server_negotiation.cc:407] Connection header received
0808 03:03:21.233522 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.233528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.233626 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.233757 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.235420 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.236208 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.237757 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.238249 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.238394 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.242568 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.242574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.242577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.242623 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.245488 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.246484 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.246489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.246491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.246625 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.247099 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.247103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.247105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.247417 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.247600 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.248359 (+ 759us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.248604 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3339,"thread_start_us":3197,"threads_started":1}
I20260808 03:03:21.250123 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.229054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:42153 (local address 127.0.0.1:49320)
0808 03:03:21.231218 (+ 2164us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.231268 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:21.233212 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:21.234045 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:21.234055 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:21.234612 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:21.235180 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.235198 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.236396 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.236400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.237602 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.237615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.238509 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.238519 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.238764 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.239537 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:21.239561 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:21.242289 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:21.245665 (+ 3376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.245676 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.245696 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.246318 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.246753 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.246759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.246763 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.246960 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.247559 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:21.247569 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:21.247969 (+ 400us) client_negotiation.cc:769] Sending connection context
0808 03:03:21.248258 (+ 289us) client_negotiation.cc:240] Negotiation successful
0808 03:03:21.248830 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1844,"thread_start_us":1632,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.205.188@KRBTEST.COM" created.
I20260808 03:03:21.251976 2686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:03:21.269307 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.253049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48456 (local address 127.0.205.189:39043)
0808 03:03:21.253796 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.253808 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.253843 (+ 35us) server_negotiation.cc:407] Connection header received
0808 03:03:21.253941 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.253954 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.254210 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.254363 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.255085 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.256799 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.258606 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.259152 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.259260 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.263137 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.263151 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.263166 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.263201 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.266210 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.266893 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.266906 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.266911 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.267079 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.267585 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.267593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.267598 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.268118 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.268367 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.268729 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.269006 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":544,"thread_start_us":392,"threads_started":1}
I20260808 03:03:21.269392 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.252876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48456)
0808 03:03:21.253073 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.253093 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:21.253210 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:21.254385 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:21.254389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:21.254749 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:21.254910 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.254919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.256990 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.256998 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.258456 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.258474 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.259313 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.259325 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.259541 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.260205 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:21.260233 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:21.262919 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:21.266346 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.266351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.266356 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.266720 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.267238 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.267243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.267246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.267429 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.268283 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:21.268286 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:21.268407 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 03:03:21.268517 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 03:03:21.268898 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 03:03:21.272284 2686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:03:21.273453 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.273010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:41680)
0808 03:03:21.273171 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.273288 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 03:03:21.273693 2614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
I20260808 03:03:21.275292 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.274957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:41696)
0808 03:03:21.275116 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.275192 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 03:03:21.275503 2686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:42707: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.205.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab.
Entry for principal kudu/127.0.205.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab.
I20260808 03:03:21.298557 822 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
/tmp/dist-test-taskvzvYzW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.188:42707
--webserver_interface=127.0.205.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab
--principal=kudu/127.0.205.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.205.148:45259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.205.190:42153,127.0.205.189:39043,127.0.205.188:42707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 03:03:21.324854 2686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } attempt: 1
I20260808 03:03:21.326234 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.325673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:41702)
0808 03:03:21.325966 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.326075 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 03:03:21.327405 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.327058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:41712)
0808 03:03:21.327205 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.327294 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 03:03:21.327732 2686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:42707: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
I20260808 03:03:21.363528 2571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } attempt: 5
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:03:21.378708 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.364611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.0.205.189:39043)
0808 03:03:21.364806 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.364811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.364833 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:03:21.364893 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.364897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.364959 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.365045 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.366158 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.367073 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.368728 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.369165 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.369261 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.373150 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.373154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.373157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.373191 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.375881 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.376766 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.376770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.376772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.376951 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.377385 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.377388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.377390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.377664 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.377776 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.378286 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.378481 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 03:03:21.378854 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.364260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:48462)
0808 03:03:21.364548 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.364573 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:21.364735 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:21.365129 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:21.365134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:21.365660 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:21.365996 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.366012 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.367212 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.367218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.368581 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.368597 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.369324 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.369331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.369563 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.370283 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:21.370307 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:21.372950 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:21.376029 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.376034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.376038 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.376626 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.377084 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.377089 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.377093 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.377268 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.377793 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:21.377798 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:21.378010 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 03:03:21.378193 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 03:03:21.378509 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260808 03:03:21.379966 2571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:03:21.381003 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.380633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:41726)
0808 03:03:21.380797 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.380874 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 03:03:21.382004 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.381605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:41738)
0808 03:03:21.381751 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.381834 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 03:03:21.382226 2571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:42707: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
I20260808 03:03:21.396142 2686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } attempt: 2
I20260808 03:03:21.397544 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.396936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56170)
0808 03:03:21.397150 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.397313 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 03:03:21.398586 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.398240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56178)
0808 03:03:21.398391 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.398470 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 03:03:21.398785 2686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:42707: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
I20260808 03:03:21.433527 2571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } attempt: 1
I20260808 03:03:21.434815 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.434308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56190)
0808 03:03:21.434542 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.434644 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 03:03:21.435798 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.435476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56204)
0808 03:03:21.435623 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.435696 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 03:03:21.436012 2571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:42707: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
I20260808 03:03:21.490049 2686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } attempt: 3
I20260808 03:03:21.491340 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.490857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56212)
0808 03:03:21.491087 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.491182 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 03:03:21.492450 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.492016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56220)
0808 03:03:21.492261 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.492335 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260808 03:03:21.492647 2686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:42707: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
I20260808 03:03:21.504227 2571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } attempt: 2
I20260808 03:03:21.505405 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.504908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56232)
0808 03:03:21.505130 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.505225 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 03:03:21.506423 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.506098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56238)
0808 03:03:21.506244 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.506317 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 03:03:21.506614 2571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:42707: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
I20260808 03:03:21.612826 2571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } attempt: 3
I20260808 03:03:21.614215 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.613560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56254)
0808 03:03:21.613840 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.613983 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260808 03:03:21.615536 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.614962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56256)
0808 03:03:21.615244 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.615370 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260808 03:03:21.615815 2571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:42707: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
I20260808 03:03:21.635906 2686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } attempt: 4
I20260808 03:03:21.637377 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.636703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56260)
0808 03:03:21.637013 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.637147 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260808 03:03:21.638676 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.638239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56262)
0808 03:03:21.638427 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.638525 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 03:03:21.638994 2686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.205.188:42707: Network error: Client connection negotiation failed: client connection to 127.0.205.188:42707: connect: Connection refused (error 111)
W20260808 03:03:21.675942 2702 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 03:03:21.676313 2702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 03:03:21.676434 2702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 03:03:21.676559 2702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 03:03:21.684123 2702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 03:03:21.684214 2702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 03:03:21.684314 2702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 03:03:21.684376 2702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 03:03:21.684494 2702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 03:03:21.694479 2702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.205.148:45259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.205.190:42153,127.0.205.189:39043,127.0.205.188:42707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/krb5kdc/kudu_127.0.205.188.keytab
--principal=kudu/127.0.205.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.205.188:42707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.205.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 01:54:35 UTC on ccycloud.quasar-csseqe.root.comops.site
build id 4574015
ASAN enabled
I20260808 03:03:21.696085 2702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 03:03:21.697582 2702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 03:03:21.709700 2708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:21.709812 2710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 03:03:21.709775 2707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 03:03:21.712656 2702 server_base.cc:1034] running on GCE node
I20260808 03:03:21.713636 2702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 03:03:21.715390 2702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 03:03:21.716620 2702 hybrid_clock.cc:648] HybridClock initialized: now 1786158201716558 us; error 31 us; skew 500 ppm
I20260808 03:03:21.716986 2702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 03:03:21.720508 2702 init.cc:377] Logged in from keytab as kudu/127.0.205.188@KRBTEST.COM (short username kudu)
I20260808 03:03:21.722180 2702 webserver.cc:460] Webserver started at http://127.0.205.188:40755/ using document root <none> and password file <none>
I20260808 03:03:21.723336 2702 fs_manager.cc:359] Metadata directory not provided
I20260808 03:03:21.723480 2702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 03:03:21.723788 2702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 03:03:21.727106 2702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/data/instance:
uuid: "b382e90ce94049b6b0953459815ac92a"
format_stamp: "Formatted at 2026-08-08 03:03:21 on dist-test-slave-6ntk"
server_key: "b47dc254cb81874eeced5d8a01e8618d"
server_key_iv: "cb1c8949592cf3930fed72f83a3bdf12"
server_key_version: "encryptionkey@0"
I20260808 03:03:21.728001 2702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/wal/instance:
uuid: "b382e90ce94049b6b0953459815ac92a"
format_stamp: "Formatted at 2026-08-08 03:03:21 on dist-test-slave-6ntk"
server_key: "b47dc254cb81874eeced5d8a01e8618d"
server_key_iv: "cb1c8949592cf3930fed72f83a3bdf12"
server_key_version: "encryptionkey@0"
I20260808 03:03:21.733414 2702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 03:03:21.737146 2720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 03:03:21.738706 2702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 03:03:21.738906 2702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/data,/tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/wal
uuid: "b382e90ce94049b6b0953459815ac92a"
format_stamp: "Formatted at 2026-08-08 03:03:21 on dist-test-slave-6ntk"
server_key: "b47dc254cb81874eeced5d8a01e8618d"
server_key_iv: "cb1c8949592cf3930fed72f83a3bdf12"
server_key_version: "encryptionkey@0"
I20260808 03:03:21.739164 2702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 03:03:21.751017 2702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 03:03:21.755780 2702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 03:03:21.756096 2702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 03:03:21.776437 2702 rpc_server.cc:307] RPC server started. Bound to: 127.0.205.188:42707
I20260808 03:03:21.776458 2787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.205.188:42707 every 8 connection(s)
I20260808 03:03:21.777958 2702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/data/info.pb
I20260808 03:03:21.781669 2791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 03:03:21.785561 822 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu as pid 2702
I20260808 03:03:21.785763 822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvzvYzW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786157811617362-822-0/minicluster-data/master-2/wal/instance
I20260808 03:03:21.786351 822 external_mini_cluster.cc:1428] Setting key 9e57e87ee1abad64c6c777a02bc24ba7
I20260808 03:03:21.789767 2791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:03:21.793068 2571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } attempt: 4
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.190@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:03:21.812754 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.793804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56272)
0808 03:03:21.794027 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.794052 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:21.794161 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:21.795424 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:21.795429 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:21.795951 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:21.796140 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.796152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.798271 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.798277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.799805 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.799819 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.800731 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.800737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.800915 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.801583 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:21.801605 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:21.804617 (+ 3012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:21.808989 (+ 4372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.808996 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.809000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.809802 (+ 802us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.810353 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.810361 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.810367 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.810541 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.811624 (+ 1083us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:21.811628 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:21.811822 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 03:03:21.812013 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 03:03:21.812323 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 03:03:21.813417 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.794771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45788 (local address 127.0.205.190:42153)
0808 03:03:21.795259 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.795264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.796284 (+ 1020us) server_negotiation.cc:407] Connection header received
0808 03:03:21.796528 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.796533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.796610 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.796713 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.798207 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.799366 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.801437 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.802187 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.802321 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.806219 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.806228 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.806234 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.806282 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.809776 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.810920 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.810927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.810930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.811066 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.811535 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.811543 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.811547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.811903 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.812087 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.812814 (+ 727us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.813059 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":313,"threads_started":1}
I20260808 03:03:21.814565 2796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.794102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.0.205.188:42707)
0808 03:03:21.794875 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.794883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.795005 (+ 122us) server_negotiation.cc:407] Connection header received
0808 03:03:21.795076 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.795081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.795154 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.796114 (+ 960us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.796590 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.798128 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.799963 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.800557 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.800696 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.804864 (+ 4168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.804872 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.804877 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.804929 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.808836 (+ 3907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.809979 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.809986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.809994 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.810184 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.810695 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.810699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.810701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.811412 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.811653 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.813559 (+ 1906us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.813766 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":598,"spinlock_wait_cycles":118528,"thread_start_us":424,"threads_started":1}
I20260808 03:03:21.814625 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.788320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56268 (local address 127.0.205.188:42707)
0808 03:03:21.789208 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.789223 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.789276 (+ 53us) server_negotiation.cc:407] Connection header received
0808 03:03:21.791043 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.791070 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.791545 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.791971 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.794048 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.796535 (+ 2487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.798604 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.799278 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.799488 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.804551 (+ 5063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.804562 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.804579 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.804614 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.809010 (+ 4396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.809955 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.809965 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.809974 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.810169 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.810639 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.810644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.810649 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.811425 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.811701 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.812935 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.813191 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":658,"spinlock_wait_cycles":23424,"thread_start_us":408,"threads_started":1}
I20260808 03:03:21.814663 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.794148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.190:42153 (local address 127.0.0.1:45788)
0808 03:03:21.796050 (+ 1902us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.796096 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:21.796352 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:21.796904 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:21.796909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:21.797724 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:21.798059 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.798069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.799590 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.799595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.801249 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.801261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.802453 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.802464 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.802634 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.803255 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:21.803291 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:21.806008 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:21.810271 (+ 4263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.810278 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.810283 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.810767 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.811184 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.811189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.811192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.811398 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.812042 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:21.812051 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:21.812250 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 03:03:21.812504 (+ 254us) client_negotiation.cc:240] Negotiation successful
0808 03:03:21.813068 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1693,"spinlock_wait_cycles":55424,"thread_start_us":822,"threads_started":1}
I20260808 03:03:21.816404 2791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:03:21.823398 2571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44: Bootstrap starting.
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:03:21.827687 2571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44: Neither blocks nor log segments found. Creating new log.
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.190@KRBTEST.COM
I20260808 03:03:21.828895 2571 log.cc:824] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44: Log is configured to *not* fsync() on all Append() calls
I20260808 03:03:21.832343 2571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44: No bootstrap required, opened a new log
I20260808 03:03:21.833349 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.817240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60642 (local address 127.0.205.189:39043)
0808 03:03:21.817436 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.817443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.817466 (+ 23us) server_negotiation.cc:407] Connection header received
0808 03:03:21.817535 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.817542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.817640 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.817768 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.818613 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.819624 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.820781 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.821486 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.821607 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.825630 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.825640 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.825647 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.825699 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.829352 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.830292 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.830298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.830302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.830497 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.831033 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.831040 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.831044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.831454 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.831599 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.832650 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.832925 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:03:21.834379 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.817110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.189:39043 (local address 127.0.0.1:60642)
0808 03:03:21.817296 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.817321 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:21.817405 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:21.817803 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:21.817807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:21.818317 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:21.818459 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.818467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.819771 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.819779 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.820639 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.820650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.821656 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.821662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.821861 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.822602 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:21.822629 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:21.825375 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:21.829497 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.829503 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.829509 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.830144 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.830682 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.830687 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.830691 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.830874 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.831693 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:21.831700 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:21.832319 (+ 619us) client_negotiation.cc:769] Sending connection context
0808 03:03:21.833634 (+ 1315us) client_negotiation.cc:240] Negotiation successful
0808 03:03:21.833983 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 03:03:21.835364 2791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } has no permanent_uuid. Determining permanent_uuid...
I20260808 03:03:21.836828 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.818847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.0.205.190:42153)
0808 03:03:21.819023 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.819029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.819047 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:03:21.819125 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.819129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.819222 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.819315 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.820856 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.821941 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.823362 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.823876 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.824013 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.829121 (+ 5108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.829128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.829133 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.829173 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.833132 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.833983 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.833989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.833992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.834161 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.834616 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.834621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.834626 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.835045 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.835357 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.835964 (+ 607us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.836255 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:03:21.842957 2571 raft_consensus.cc:348] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } }
I20260808 03:03:21.843451 2571 raft_consensus.cc:729] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a62d96eeda34413da9abda08d956cd44, State: Initialized, Role: FOLLOWER
I20260808 03:03:21.844056 2571 consensus_queue.cc:260] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } }
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.188@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
I20260808 03:03:21.846306 2804 sys_catalog.cc:455] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } } }
I20260808 03:03:21.846725 2571 sys_catalog.cc:564] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:21.846927 2804 sys_catalog.cc:458] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.205.189@KRBTEST.COM
I20260808 03:03:21.852057 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.836100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56282)
0808 03:03:21.836735 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.836753 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:21.836907 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:21.837292 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:21.837295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:21.837837 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:21.838115 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.838129 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.839443 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.839450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.840822 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.840837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.841788 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.841795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.842037 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.842800 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:21.842826 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:21.845522 (+ 2696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:21.849413 (+ 3891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.849420 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.849425 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.849823 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.850324 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.850331 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.850334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.850473 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.851003 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:21.851008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:21.851150 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 03:03:21.851297 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 03:03:21.851640 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":3}
I20260808 03:03:21.853039 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.836468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.0.205.188:42707)
0808 03:03:21.836643 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.836650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.836804 (+ 154us) server_negotiation.cc:407] Connection header received
0808 03:03:21.837036 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.837040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.837108 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.837155 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.838292 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.839300 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.840971 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.841622 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.841756 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.845723 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.845731 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.845736 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.845775 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.849243 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.850076 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.850086 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.850088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.850211 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.850595 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.850600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.850603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.850877 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.851043 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.851389 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.852660 (+ 1271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1000,"server-negotiator.queue_time_us":62}
I20260808 03:03:21.858985 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.839189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.0.205.189:39043)
0808 03:03:21.839351 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.839358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.839375 (+ 17us) server_negotiation.cc:407] Connection header received
0808 03:03:21.839436 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.839443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.839522 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.839629 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.840905 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.841997 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.843993 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.844781 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.844921 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.851591 (+ 6670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.851600 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.851605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.851650 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.855358 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.856306 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.856312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.856316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.856499 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.857098 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.857103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.857107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.857453 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.857736 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.858257 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.858528 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 03:03:21.859604 2791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a: Bootstrap starting.
I20260808 03:03:21.863942 2791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:21.865187 2791 log.cc:824] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a: Log is configured to *not* fsync() on all Append() calls
I20260808 03:03:21.867799 822 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 03:03:21.869145 2822 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:03:21.869268 2822 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:03:21.870204 2791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a: No bootstrap required, opened a new log
I20260808 03:03:21.870746 2524 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a62d96eeda34413da9abda08d956cd44"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45804
I20260808 03:03:21.871232 2524 raft_consensus.cc:482] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 03:03:21.871454 2524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:21.875259 2524 raft_consensus.cc:504] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } }
I20260808 03:03:21.867511 2463 master_runner.cc:186] Error getting master registration for 127.0.205.189:39043: OK, instance_id { permanent_uuid: "895a376384614816875777d20ee7ba58" instance_seqno: 1786158201206835 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:03:21.879015 2524 leader_election.cc:288] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [CANDIDATE]: Term 1 election: Requested vote from peers 895a376384614816875777d20ee7ba58 (127.0.205.189:39043), b382e90ce94049b6b0953459815ac92a (127.0.205.188:42707)
I20260808 03:03:21.879423 2463 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:03:21.881326 2791 raft_consensus.cc:348] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } }
I20260808 03:03:21.881819 2791 raft_consensus.cc:729] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b382e90ce94049b6b0953459815ac92a, State: Initialized, Role: FOLLOWER
I20260808 03:03:21.882447 2791 consensus_queue.cc:260] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } }
W20260808 03:03:21.887892 2493 tablet.cc:2369] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:03:21.889465 2831 sys_catalog.cc:455] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } } }
I20260808 03:03:21.890017 2831 sys_catalog.cc:458] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:21.892182 2791 sys_catalog.cc:564] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:21.892249 2752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a62d96eeda34413da9abda08d956cd44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b382e90ce94049b6b0953459815ac92a"
I20260808 03:03:21.892795 2752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:21.898231 2752 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a62d96eeda34413da9abda08d956cd44 in term 1.
I20260808 03:03:21.899396 2490 leader_election.cc:302] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a62d96eeda34413da9abda08d956cd44, b382e90ce94049b6b0953459815ac92a; no voters:
I20260808 03:03:21.900005 2804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 03:03:21.916474 2804 raft_consensus.cc:686] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [term 1 LEADER]: Becoming Leader. State: Replica: a62d96eeda34413da9abda08d956cd44, State: Running, Role: LEADER
I20260808 03:03:21.917214 2804 consensus_queue.cc:237] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } }
I20260808 03:03:21.920050 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.887211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.0.205.189:39043)
0808 03:03:21.887590 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.887596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.887813 (+ 217us) server_negotiation.cc:407] Connection header received
0808 03:03:21.888541 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.888551 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.888667 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.888765 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.901129 (+ 12364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.902703 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.905500 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.906459 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.906603 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.911246 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.911257 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.911264 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.911329 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.916035 (+ 4706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.917235 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.917243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.917247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.917435 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.918132 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.918137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.918142 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.918527 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.918756 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.919429 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.919691 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260808 03:03:21.929088 2837 sys_catalog.cc:455] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a62d96eeda34413da9abda08d956cd44. Latest consensus state: current_term: 1 leader_uuid: "a62d96eeda34413da9abda08d956cd44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } } }
I20260808 03:03:21.929647 2837 sys_catalog.cc:458] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:21.935858 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.887026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45820 (local address 127.0.205.190:42153)
0808 03:03:21.887419 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.887427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.902489 (+ 15062us) server_negotiation.cc:407] Connection header received
0808 03:03:21.902777 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.902783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.902905 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.902989 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.907000 (+ 4011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.908426 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.915374 (+ 6948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.916367 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.916505 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.922513 (+ 6008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.922520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.922524 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.922568 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.927261 (+ 4693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.930026 (+ 2765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.930033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.930037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.930206 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.931000 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.931033 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.931037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.931783 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.932329 (+ 546us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.933314 (+ 985us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.935423 (+ 2109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260808 03:03:21.937361 2844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 03:03:21.938215 2686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } attempt: 5
I20260808 03:03:21.939872 2844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 03:03:21.946695 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.902869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56288 (local address 127.0.205.188:42707)
0808 03:03:21.903075 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.903083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.921349 (+ 18266us) server_negotiation.cc:407] Connection header received
0808 03:03:21.921658 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.921666 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.921769 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.921834 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.923731 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.925053 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.927423 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.928067 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.928190 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.932184 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.932193 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.932199 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.932251 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.936701 (+ 4450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.937741 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.937750 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.937756 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.938002 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.941722 (+ 3720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.941757 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.941763 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.942577 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.943714 (+ 1137us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.944301 (+ 587us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.945219 (+ 918us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 03:03:21 dist-test-slave-6ntk krb5kdc[2447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786158201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.205.189@KRBTEST.COM for kudu/127.0.205.188@KRBTEST.COM
I20260808 03:03:21.965044 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.938881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.205.188:42707 (local address 127.0.0.1:56298)
0808 03:03:21.939362 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 03:03:21.939402 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 03:03:21.939545 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 03:03:21.940588 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 03:03:21.940594 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 03:03:21.941146 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 03:03:21.941373 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.941389 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.943431 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.943440 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.945445 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 03:03:21.945469 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.947173 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.947180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 03:03:21.948588 (+ 1408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.949500 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 03:03:21.949534 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 03:03:21.953838 (+ 4304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 03:03:21.960946 (+ 7108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.960955 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.960962 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.961749 (+ 787us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.962346 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 03:03:21.962354 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 03:03:21.962361 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 03:03:21.962588 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 03:03:21.963574 (+ 986us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 03:03:21.963583 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 03:03:21.963813 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 03:03:21.964010 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 03:03:21.964464 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317}
I20260808 03:03:21.966641 2796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:21.939796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56298 (local address 127.0.205.188:42707)
0808 03:03:21.940077 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:21.940086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:21.940108 (+ 22us) server_negotiation.cc:407] Connection header received
0808 03:03:21.940172 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:21.940181 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:21.940276 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:21.940414 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:21.941571 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.943142 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.945718 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:21.946774 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:21.946923 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:21.954178 (+ 7255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 03:03:21.954189 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 03:03:21.954197 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 03:03:21.954250 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 03:03:21.959682 (+ 5432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.961956 (+ 2274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.961964 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.961971 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.962163 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 03:03:21.962751 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 03:03:21.962760 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 03:03:21.962765 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 03:03:21.963268 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 03:03:21.963475 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:21.965570 (+ 2095us) server_negotiation.cc:299] Negotiation successful
0808 03:03:21.965949 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 03:03:21.974887 2686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58: Bootstrap starting.
I20260808 03:03:21.981681 2686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58: Neither blocks nor log segments found. Creating new log.
I20260808 03:03:21.983563 2686 log.cc:824] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58: Log is configured to *not* fsync() on all Append() calls
I20260808 03:03:21.991676 2752 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [term 1 FOLLOWER]: Refusing update from remote peer a62d96eeda34413da9abda08d956cd44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:03:21.992638 2804 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:03:21.993484 2686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58: No bootstrap required, opened a new log
I20260808 03:03:22.006704 2686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } }
I20260808 03:03:22.007314 2686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 895a376384614816875777d20ee7ba58, State: Initialized, Role: FOLLOWER
W20260808 03:03:22.008268 2727 tablet.cc:2369] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 03:03:22.007944 2686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } }
I20260808 03:03:22.011430 2831 sys_catalog.cc:455] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a62d96eeda34413da9abda08d956cd44. Latest consensus state: current_term: 1 leader_uuid: "a62d96eeda34413da9abda08d956cd44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } } }
I20260808 03:03:22.011757 2831 sys_catalog.cc:458] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:22.016222 2859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } } }
I20260808 03:03:22.016669 2859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:22.017436 2851 mvcc.cc:205] Tried to move back new op lower bound from 7316103995345416192 to 7316103995059949568. Current Snapshot: MvccSnapshot[applied={T|T < 7316103995345416192}]
I20260808 03:03:22.018610 2686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 03:03:22.018514 2831 sys_catalog.cc:455] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a62d96eeda34413da9abda08d956cd44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } } }
I20260808 03:03:22.018774 2831 sys_catalog.cc:458] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:22.018682 2643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a62d96eeda34413da9abda08d956cd44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "895a376384614816875777d20ee7ba58"
I20260808 03:03:22.019233 2643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [term 0 FOLLOWER]: Advancing to term 1
I20260808 03:03:22.024756 2643 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a62d96eeda34413da9abda08d956cd44 in term 1.
I20260808 03:03:22.028784 2642 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [term 1 FOLLOWER]: Refusing update from remote peer a62d96eeda34413da9abda08d956cd44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 03:03:22.030714 2804 sys_catalog.cc:455] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a62d96eeda34413da9abda08d956cd44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } } }
I20260808 03:03:22.031121 2804 sys_catalog.cc:458] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:22.032150 2837 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 03:03:22.033440 2844 catalog_manager.cc:1347] Generated new cluster ID: 84168ea55802458caf2196ec8ee08d9d
I20260808 03:03:22.033557 2844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260808 03:03:22.038591 2860 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 03:03:22.038767 2860 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 03:03:22.045302 2702 master_runner.cc:186] Error getting master registration for 127.0.205.189:39043: OK, instance_id { permanent_uuid: "895a376384614816875777d20ee7ba58" instance_seqno: 1786158201206835 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 03:03:22.058990 2702 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 03:03:22.073500 2859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a62d96eeda34413da9abda08d956cd44. Latest consensus state: current_term: 1 leader_uuid: "a62d96eeda34413da9abda08d956cd44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } } }
I20260808 03:03:22.074084 2859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:22.075191 2875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a62d96eeda34413da9abda08d956cd44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } } }
I20260808 03:03:22.075738 2875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 03:03:22.080186 2804 sys_catalog.cc:455] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a62d96eeda34413da9abda08d956cd44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a62d96eeda34413da9abda08d956cd44" member_type: VOTER last_known_addr { host: "127.0.205.190" port: 42153 } } peers { permanent_uuid: "895a376384614816875777d20ee7ba58" member_type: VOTER last_known_addr { host: "127.0.205.189" port: 39043 } } peers { permanent_uuid: "b382e90ce94049b6b0953459815ac92a" member_type: VOTER last_known_addr { host: "127.0.205.188" port: 42707 } } }
I20260808 03:03:22.080514 2804 sys_catalog.cc:458] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44 [sys.catalog]: This master's current role is: LEADER
I20260808 03:03:22.082823 2844 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 03:03:22.085244 2844 catalog_manager.cc:1504] Loading token signing keys...
I20260808 03:03:22.105190 2882 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 03:03:22.109499 2844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a62d96eeda34413da9abda08d956cd44: Generated new TSK 0
I20260808 03:03:22.111307 2844 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 03:03:22.196347 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.184860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.0.205.189:39043)
0808 03:03:22.185081 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.185087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.191000 (+ 5913us) server_negotiation.cc:407] Connection header received
0808 03:03:22.191339 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.191349 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.191604 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.191760 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.195036 (+ 3276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60662: BlockingRecv error: recv got EOF from 127.0.0.1:60662 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 03:03:22.197130 2801 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.184174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45836 (local address 127.0.205.190:42153)
0808 03:03:22.184643 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.184653 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.194985 (+ 10332us) server_negotiation.cc:407] Connection header received
0808 03:03:22.195286 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.195292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.195385 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.195468 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.196869 (+ 1401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45836: BlockingRecv error: recv got EOF from 127.0.0.1:45836 (error 108)
Metrics: {"server-negotiator.queue_time_us":289}
W20260808 03:03:22.201289 2796 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.189973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56314 (local address 127.0.205.188:42707)
0808 03:03:22.190299 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.190305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.197891 (+ 7586us) server_negotiation.cc:407] Connection header received
0808 03:03:22.198070 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.198078 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.198203 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.198376 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.200995 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56314: BlockingRecv error: recv got EOF from 127.0.0.1:56314 (error 108)
Metrics: {"server-negotiator.queue_time_us":161}
W20260808 03:03:22.208045 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.205751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60666 (local address 127.0.205.189:39043)
0808 03:03:22.206005 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.206014 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.206527 (+ 513us) server_negotiation.cc:407] Connection header received
0808 03:03:22.206792 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.206796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.206904 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.207025 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.207774 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60666: BlockingRecv error: recv got EOF from 127.0.0.1:60666 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 03:03:22.226120 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.213100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60668 (local address 127.0.205.189:39043)
0808 03:03:22.213277 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.213283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.224050 (+ 10767us) server_negotiation.cc:407] Connection header received
0808 03:03:22.224585 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.224592 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.224687 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.224759 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.225856 (+ 1097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60668: BlockingRecv error: recv got EOF from 127.0.0.1:60668 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 03:03:22.262163 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.259453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60676 (local address 127.0.205.189:39043)
0808 03:03:22.259684 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.259690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.260582 (+ 892us) server_negotiation.cc:407] Connection header received
0808 03:03:22.260829 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.260834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.260910 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.261025 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.261853 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60676: BlockingRecv error: recv got EOF from 127.0.0.1:60676 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 03:03:22.271942 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.269528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60684 (local address 127.0.205.189:39043)
0808 03:03:22.269808 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.269817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.270413 (+ 596us) server_negotiation.cc:407] Connection header received
0808 03:03:22.270642 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.270649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.270729 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.270850 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.271690 (+ 840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60684: BlockingRecv error: recv got EOF from 127.0.0.1:60684 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
W20260808 03:03:22.285959 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.283526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.0.205.189:39043)
0808 03:03:22.283792 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.283801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.284382 (+ 581us) server_negotiation.cc:407] Connection header received
0808 03:03:22.284637 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.284642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.284721 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.284854 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.285641 (+ 787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60698: BlockingRecv error: recv got EOF from 127.0.0.1:60698 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 03:03:22.311833 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.308187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.0.205.189:39043)
0808 03:03:22.308455 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.308464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.310279 (+ 1815us) server_negotiation.cc:407] Connection header received
0808 03:03:22.310556 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.310561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.310649 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.310769 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.311584 (+ 815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60702: BlockingRecv error: recv got EOF from 127.0.0.1:60702 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 03:03:22.350011 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.347603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.0.205.189:39043)
0808 03:03:22.347828 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.347837 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.348442 (+ 605us) server_negotiation.cc:407] Connection header received
0808 03:03:22.348696 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.348702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.348780 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.348901 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.349681 (+ 780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60710: BlockingRecv error: recv got EOF from 127.0.0.1:60710 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260808 03:03:22.427095 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.424200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.0.205.189:39043)
0808 03:03:22.424426 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.424434 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.425156 (+ 722us) server_negotiation.cc:407] Connection header received
0808 03:03:22.425387 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.425394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.425523 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.425687 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.426828 (+ 1141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60722: BlockingRecv error: recv got EOF from 127.0.0.1:60722 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260808 03:03:22.566591 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.564100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.0.205.189:39043)
0808 03:03:22.564354 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.564361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.564675 (+ 314us) server_negotiation.cc:407] Connection header received
0808 03:03:22.564947 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.564951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.565041 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.565130 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.566201 (+ 1071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60736: BlockingRecv error: recv got EOF from 127.0.0.1:60736 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260808 03:03:22.828015 2698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 03:03:22.826107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.0.205.189:39043)
0808 03:03:22.826343 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:22.826349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:22.826512 (+ 163us) server_negotiation.cc:407] Connection header received
0808 03:03:22.826756 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:22.826769 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:22.826853 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:22.826974 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 03:03:22.827779 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60742: BlockingRecv error: recv got EOF from 127.0.0.1:60742 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 03:03:23.040665 2860 catalog_manager.cc:1259] Loaded cluster ID: 84168ea55802458caf2196ec8ee08d9d
I20260808 03:03:23.040781 2860 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a: loading cluster ID for follower catalog manager: success
I20260808 03:03:23.043532 2860 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a: acquiring CA information for follower catalog manager: success
I20260808 03:03:23.045439 2860 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b382e90ce94049b6b0953459815ac92a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 03:03:23.110349 2882 catalog_manager.cc:1259] Loaded cluster ID: 84168ea55802458caf2196ec8ee08d9d
I20260808 03:03:23.110503 2882 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58: loading cluster ID for follower catalog manager: success
I20260808 03:03:23.113220 2882 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58: acquiring CA information for follower catalog manager: success
I20260808 03:03:23.115231 2882 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 895a376384614816875777d20ee7ba58: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 03:03:23.352895 2992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:23.344450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60752 (local address 127.0.205.189:39043)
0808 03:03:23.345467 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:23.345472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:23.345490 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:03:23.345558 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:23.345561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:23.345726 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:23.345837 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:23.346705 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:23.347803 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:23.350121 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:23.350604 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:23.350757 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:23.351425 (+ 668us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:23.352084 (+ 659us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:23.352205 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:23.352412 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 03:03:23.352485 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":898,"thread_start_us":641,"threads_started":1}
I20260808 03:03:23.367427 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:23.359648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56318 (local address 127.0.205.188:42707)
0808 03:03:23.360308 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:23.360313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:23.360331 (+ 18us) server_negotiation.cc:407] Connection header received
0808 03:03:23.360530 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:23.360535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:23.360704 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:23.360853 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:23.361439 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:23.362352 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:23.364590 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:23.365104 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:23.365204 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:23.365990 (+ 786us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:23.366590 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:23.366737 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:23.366995 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 03:03:23.367079 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":442,"threads_started":1}
I20260808 03:03:23.390340 2992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:23.374729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.0.205.189:39043)
0808 03:03:23.380139 (+ 5410us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:23.380149 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:23.381928 (+ 1779us) server_negotiation.cc:407] Connection header received
0808 03:03:23.382029 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:23.382035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:23.382151 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:23.382279 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:23.383183 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:23.384542 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:23.387218 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:23.388080 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:23.388243 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:23.388967 (+ 724us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:23.389121 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:23.389217 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:23.389846 (+ 629us) server_negotiation.cc:299] Negotiation successful
0808 03:03:23.389967 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5204}
I20260808 03:03:23.397854 3012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:23.374353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.0.205.190:42153)
0808 03:03:23.387191 (+ 12838us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:23.387199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:23.387936 (+ 737us) server_negotiation.cc:407] Connection header received
0808 03:03:23.388292 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:23.388297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:23.388573 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:23.388744 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:23.389792 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:23.391179 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:23.393660 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:23.394592 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:23.394730 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:23.395780 (+ 1050us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:23.396503 (+ 723us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:23.396607 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:23.397237 (+ 630us) server_negotiation.cc:299] Negotiation successful
0808 03:03:23.397376 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12645,"thread_start_us":12851,"threads_started":1}
I20260808 03:03:23.399242 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 03:03:23.388269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56326 (local address 127.0.205.188:42707)
0808 03:03:23.388460 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 03:03:23.388466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 03:03:23.391467 (+ 3001us) server_negotiation.cc:407] Connection header received
0808 03:03:23.391802 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 03:03:23.391809 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 03:03:23.391929 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 03:03:23.392004 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 03:03:23.392668 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:23.393857 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:23.396260 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 03:03:23.396985 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 03:03:23.397102 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 03:03:23.397949 (+ 847us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 03:03:23.398121 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 03:03:23.398197 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 03:03:23.398831 (+ 634us) server_negotiation.cc:299] Negotiation successful
0808 03:03:23.398931 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 03:03:23.404083 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2463
I20260808 03:03:23.405195 2563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:03:23.520781 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 2463
W20260808 03:03:23.551046 2722 connection.cc:535] server connection from 127.0.0.1:56272 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 03:03:23.551662 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2580
I20260808 03:03:23.552906 2677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:03:23.677627 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 2580
I20260808 03:03:23.706518 822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu pid 2702
I20260808 03:03:23.707707 2783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 03:03:23.817096 822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvzvYzW/build/asan/bin/kudu with pid 2702
2026-08-08T03:03:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5384 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10703 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (392128 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST